Let me note just a few points:
1. "Holdout" doesn't mean "stubborn." A stubborn homeowner is simply someone who places a high idiosyncratic value on his home. A holdout is someone who may value his home at little more than the market value, but who sees an opportunity to obtain a much higher price by exploiting his strategic position vis-a-vis the developer. If the developer has sunk heavy costs in acquiring adjacent property, and would have to write off those costs if he can't acquire the remaining lot, then he will have to pay a high price to the owner of that lot.
2. The holdout issue is discussed at some length in several of the briefs in the Kelo case, so it's surprising that it didn't receive attention in the Court's opinions. It is unclear whether the plaintiffs were holdouts, and, as Becker mentions, even if they were there are ways of foiling holdouts without resorting to eminent domain. The simplest is to conceal the developer's project, as by using straw purchasers, so that the owners of the property that he is seeking to acquire don't know that someone is trying to put together an assemblage of contiguous lots. That is deception, but deception in a good cause, as it were, since it is countering the exploitation of a form of monopoly power.
3. I mentioned that in the absence of holding out, eminent domain operates as a form of arbitrary taxation (i.e., of idiosyncratic values). However, much taxation is arbitrary, and so the question presented by the use of eminent domain in situations not involving holding out is whether it is more arbitrary than the taxation that will be used in its stead. That is, if government land-acquisition costs are raised by limiting eminent domain, other taxes will have to be levied to pay for the higher costs.