When I blogged about the theories behind NIMBYism a few months ago (and again more recently) some of the more thoughtful comments stated, in so many words: yes, the current zoning system is sometimes abused, but certainly neighborhood concerns should be balanced against citywide interests.
From a purely libertarian perspective, of course, this argument is nonsense. A principled libertarian would suggest that Landowner A has the right to use its land as it chooses, and if nearby Landowner B doesn't like that choice, B should be willing to purchase the land itself or must tolerate the first owner's exercise of freedom.* If you follow this perspective, your use of your property is no more your neighbor's concern than is your religion or the color of your computer.
However, most people are not that libertarian, and instead prefer to rely on utilitarian considerations. From a utilitarian perspective, should cities give neighborhood preferences weight in deciding what gets built where? It depends.
For example, the current zoning system does seem to have given Americans an adequate amount of commercial space. Because businesses do not send children to school or use Medicaid, they arguably produce more tax revenue than they consume, and therefore tend to be popular with local governments. Thus, it is hard to argue that Americans suffer from a shortage of commercial space (though an urbanist or environmentalist might point out that such space is not as walkable as it should be).
It could even be argued that the current zoning system has given Americans an adequate amount of for-sale housing—in poorer cities like Detroit and Cleveland, more than the market seems to demand. Single-family homes are generally a less controversial land use than either shops or apartments, and as a result, such housing is fairly inexpensive in many metro areas. On the other hand, some of the most heavily regulated metropolitan areas still have sky-high housing costs.
But even outside New York and California, it seems to me that the current system has not produced an adequate supply of for-rent housing. The persistence of homelessness suggests that we have failed to produce enough rental housing for the destitute and the almost-destitute. And because of exploding demand and sluggish supply, rents are becoming more and more unaffordable even for the middle class; Housing Secretary Shawn Donovan states that the U.S. is in "the worst rental affordability crisis that this country has even known." For example, in Miami rents now average 43 percent of household income. Zoning clearly deters the creation of new rental housing; even in dense cities like Chicago and Vancouver, any housing other than single-family homes is illegal in most of the city. And why is that? Because of the NIMBY (Not In My Back Yard) veto: homeowners fear rental using and use government as a tool to keep it out. Otherwise, neighborhoods would be rezoned to allow apartments, and supply would keep pace with demand.
To make matters worse, the negative effects of a rental housing shortage are far worse than the negative effects of a for-sale housing shortage. If you can't afford to buy a house, you can always rent. But if you can't afford to rent, you live on the streets or in a homeless shelter.** And those who can afford to rent live much more difficult lives than they would if rents were cheaper, as the necessity to avoid homelessness crowds out every other form of human need. Moreover, the diversion of most income to rents starves the rest of the economy; every dollar spent on rent is a dollar that can't be used to enrich health insurance companies or colleges or grocery stores.
It accordingly seems to me that cities cannot be trusted to weigh the citywide interest in new housing against neighborhood concerns, for two reasons. First, the limited supply of rental housing suggests that politicians cannot resist the temptation to obey neighborhood NIMBYs. Second, the social harm from an artificially constricted rental market is greater than the social harm from an inadequate number of Targets and Wal-Marts. In other words, trusting NIMBY-influenced local governments to balance interests enough to allow an adequate supply of rental housing makes about as much sense as trusting a 10-year-old to vote or carry a gun: the chances of abuse are simply so high that a higher authority must step in.
Thus, the better rule (especially in places where housing costs are out of control) would be some kind of statewide legislation to protect local politicians from themselves, a kind of renter's Bill of Rights. I am not yet sure exactly what the right kind of law should look like. Presumably, there are some places where no housing should be allowed for environmental reasons—for example, flood plains or places contaminated with toxic waste. But as a starting point for discussion, I would suggest that (at least in high-cost central cities) apartments should generally be allowed in most places where single-family houses are allowed.
It could be argued (as a New York Times article implied) that the law of supply and demand simply doesn't apply to housing, because developers only want to build expensive units. But the supply of rich people is in fact finite, just as the supply of rich grocery shoppers or rich television purchasers is in fact finite. So even if a landowner wants to charge a premium price for housing, it cannot do so if the supply of apartments exceeds the number of rich people. This reality explains why rent in many places is less expensive than it is in New York or San Francisco; most American cities benefit from some mix of lower demand and greater supply.
*Except, presumably, in certain extreme situations, such as excessive odor.
**It could be argued that high housing costs are not related to homelessness, because destitute renters are somehow a separate market from renters as a whole. Here is a table showing homelessness rates by metro area; based on a quick look at the table, it seems to me that warm-weather cities tend to have the highest rates of homelessness (perhaps because they are more popular with transients). However, expensive cold-weather cities like New York tend to have higher rates of homelessness than cheap cold-weather cities like Buffalo and Detroit. This fact suggests that there is at least a correlation between the two, and perhaps a causal relationship.