Affordable Housing Is Two Separate Issues

How many times have you read this argument?

Liberal- "We need more government spending to create affordable housing for the poor."

Conservative- "If we just had less government intervention, our city would be affordable, just like Houston or dozens of other cities- and then we wouldn't need government subsidies."

It seems to me that both sides are right- but that both sides are addressing entirely different problems.

Here in New York where I live, affordable housing is not just a problem for the very poor who are forced into homeless shelters- it is also a problem for middle (and even upper middle) class people who are forced into suburbia by skyrocketing rents.  It seems to me that government subsidies are unlikely to solve the latter problem, because in an already highly-taxed city or state, the public would probably not tolerate the level of taxation required to make housing affordable for the majority of the city's population.  

A better remedy is to allow more private sector housing construction.  If the city's overall housing supply sufficiently increased, rents would eventually go down or at least stop rising, and the city would be more affordable for the middle class, just like our nation's relatively inexpensive inland cities.  

But even if New York's rents declined to the level of Houston or Kansas City, there would still be homeless people.  In fact: even Houston has thousands of homeless people, for the simple reason that even cheaper-than-average rents may be too expensive for people who are utterly destitute and/or incapable of taking care of themselves.  These people will need public housing or significant government subsidies to be housed.   

In sum, the issue of "affordable housing" is two separate and very different problems- housing for the middle class and housing for the poor.  It seems to me that the remedies for one problem will not work for the other.