Evidence-Based Urban Planning

In a field such as planning that is rich with quantifiable data, why there so little focus on evidence rather than opinion?, wonders researcher Martin Laplante.

If you do a web search for the term "evidence-based medicine", you will find 1.5 million hits. "Evidence-based religion" gives 10,000 hits, surprising for a field where faith is so important. "Evidence-based urban planning" gives only 4 hits.

Why, in a field where so many of the objectives are quantifiable, is there so little focus on evidence rather than opinion? It's not that planning is lacking in quantification. Densities, square feet of floor space, number of parking spots, distance between certain uses, all of these get specified to three decimal points, but what seems to be lacking is evidence that changing these features of cities will achieve a measurable objective, while later evaluation to confirm whether they have achieved these objectives is typically not required.

Instead, the profession uses articles of faith based on data or analysis that is frankly insufficient to draw hard conclusions or that is extrapolated beyond what the original study might have established. For instance many studies have demonstrated that on average those who live in areas of higher population or unit density tend to drive less than those who live in lower density. What has less evidence to support it is to what degree increasing the density of an existing area will decrease driving and increase transit use. Even if density change has a predictable, measurable effect for marginal changes to low-density areas which take them from below to above the "transit-supporting" threshold, is there evidence that analogous changes will have an effect on medium and on high-density areas? Is there evidence that decreasing VMT in one area doesn't cause an increase in VMT in other areas?

After it's built, will anyone evaluate if the planning was effective?
Image courtesy of Flickr user Eli Brown.

I'm not saying that popular conclusions regarding the relationship between density and vehicle use are correct or incorrect, merely that I have seen no methodology for diagnosing and treating deviations from vehicle use objectives that uses evidence which is sufficient to support one decision rather than another. Planetizen readers are familiar with the use of a carefully selected published figure, outside the context from which it was taken, in order to justify a previously-held opinion. Density is controversial, but it's only one example among a great many articles of faith that are used to make planning decisions. LEED and LEED-ND are filled with numbers that are essentially an arbitrary consensus of a range of opinions with little solid scientific basis. Space precludes enumerating other numeric or point-based systems whose numbers are more opinion than evidence.

In my opinion, using assumptions and extrapolations to support large-scale planning policies and not using reliable predictions of scenarios nor evaluation of those predictions after the fact are common in part because urban systems are hugely complex and difficult to predict and in part because the planning process is largely political (not meant in a pejorative sense) and doesn't expect accurate predictions.

Traffic planning is an exception, of course. When the road system is not performing precisely as expected, hundreds of thousands of people who experience the discrepancy firsthand every day will demand an immediate change. Other aspects of urban planning do not have this immediately visible cause and effect. Traffic planning also tends to be carried out by engineers. They will gladly collect and crunch truckloads of data, predicting outcomes to within seconds and land required within inches, and then monitoring afterwards.

Even traffic planning models only hold as long as we keep all external variables, like buildings, uses, and economic factors, constant. Land use planning is not only all about these external factors, but about decisions made by a million individuals, which can take years to be translated into action. Without a microsimulation model of the players encompassing all of these interconnected factors, like the University of Washington's UrbanSim for instance, and trainloads of data and tuning, there is no hope of being sufficiently predictive to inform policy decisions.

The other problem is that urban planning consists of many disparate decisions years apart. The path through the cascading decisions from a comprehensive plan through to zoning and permits replace the issues of achieving specific objectives to ones of simple conformity with the letter of a document that was designed to be flexible and vague enough to avoid seeming arbitrary. None of the individual decisions are evaluated for achievement of overall objectives because no one knows in advance how many individual projects will be built and which of the many permitted choices will be selected. Overall objectives tend to be stated in the top level plans, the ones that have little ability to influence the individual decisions that will achieve those objectives.

When a cluster of road deaths or injuries is detected, transportation planners are notified and expected to fix the problem. But when the mental or physical health of individuals is affected by built form and distribution of uses, by pollution, the absence of safe places for children to play or the lack of jobs or affordable housing or good schools, no one thinks of holding urban planners to account, even if objectives were clearly spelled out in planning documents and not achieved. Expectations of predictability are so low and responsibility so diffuse that achieving objectives, if they are even measured, is an unexpected surprise. More often than not the planning documents were based on the fashionable beliefs of previous decades, about which current planners know little except that they were mistaken.

It's time to build an evidence-based urban planning discipline. It's a transformation of the profession whose time has come.

Martin Laplante, PhD, is Vice-President of RES Policy Research, an Ottawa-based consulting company. His consulting for municipal clients and business and community groups focuses on quantitative studies, models, and technology. He has been neglecting his blog, Reverse Zone, lately.



human motivation is difficult to data-ify

Just starting out in planning, a layman's observation I've made and have difficulty understanding completely is what drives human motivation. Motivation of almost anything. And motivating people to act and behave in a certain way seems to be at the foundation of what many planning initiatives are trying to flesh out. However, humans don't always behave like static logic variables. It's nearly impossible to say that "if this, then that" when humans are the ones being discussed. So planners are left with faith-based projections on loosely interpreted data while standing toe-to-toe with "balance-sheet" minded opposition. The nature of our industry is much more difficult to boil down to a predictable metric like commerce.

why not?

It seems that the only thing needed to forge an evidence based dataset to support "good" planning is a set of objectives to be measured. However, the set of objectives seems to change faster than the "fashionable beliefs of previous decades." The grass always seems greener in someone else's city (or at least in Portland) and it appears that what we desire is what we don't currently have.

There are two issues at the heart of establishing decent objectives:

1. IT WILL NEVER BE GOOD ENOUGH!! When is it ok to check off a box and say that we've met one goal and need to move on to another? I heard the FL Secretary of Transportation last week compare the number of traffic fatalities in 1960 (2,100) to the total in 2009 (2,500) and then declare that we haven't done enough. Although no deaths would be desirable, it probably isn't reasonably attainable and I'd say that we need to move on now.

2. Not everyone likes or needs the same things, or even the same things at the same time. I would like to walk everywhere, except in August, when I would prefer airport style conveyor belts encased in permanent air conditioning everywhere people exist (or at least uniformly throughout Central Florida). I heard our local op ed writer intone the marvelous benefits of a connected street network where people use neighborhood streets to get where they need to be, except his own neighborhood street, where he would prefer that people get out and push their cars until they have passed his neighborhood. The same business people that are delighted with a "vibrant downtown" wouldn't be caught dead there after 6 (when the goth-vampires appear), because they prefer the local baseball diamond and basketball hoop in the cul-de-sac with their kids. If they actually do like the "vibrant downtown" after 6, I might not want to know because it might call into question their business sense or sleeping habits.

My point is, looking at the minutia of the process that gets us to planning decisions probably doesn't get us any closer to evidence based planning, though some of those small things may have significant impacts. What is the evidence of good planning that we are seeking? Contentment, well-suitedness, employment stability, generational stability, trendiness, etc? I'm sure that for each value, we can find a planning process that statistically influenced its establishment.

We cannot, however, be all things to all people, all the time and far too often that is the real objective.

Patricia C. Tice, PE, AICP, LEED AP
AECOM D+P; Orlando, FL

Evidence based planning

Hi Martin:

Excellent article. I do appreciate that you have time to participate in EnFamille while engaging so actively in your own consulting field.

Now to me it should not be that difficult to gather evidence about how much and where people travel. It seems for political polls, they survey less than 3% of the people to make an accurate prediction. Well maybe we only need to survey 3% of the postal codes to find out what is going on.

I suppose what we would want to know is what factors are important i.e.
- proximity to place of work
- proximity to good shopping
- proximity to church
- proximity to entertainment
- proximity to school/university
- proximity to friends/relatives
- distance to weekend cottage/chalet
- etc.
So the survey could ask about this and the related miles driven. And this could be checked against the design assumptions. If the design assumptions are proven wrong then the new information could be fed back into the design parameters.

There are other issues. People might be willing to ride the subway/street cars to go to work or school but they may insist on driving their car for shopping, going to church or visiting a friend/relative. Also some people have jobs that take them from one customer to another whereby they must use their car even though they live near a subway stop.

On top of that young people will be willing to ride a bicycle to get around but older people may not.

It seems very difficult to get all this stuff quantified accurately. And it seems more reasonable to use these types of factors somewhat broadly because they will keep changing as the people change. I don't think Urban Planning can ever be reduced to tight formulas or algorithms because of the changing situations of the people and the turnover of the people who live in the neighborhoods.

Mike Smith

Public Health

I agree completely with the need to planning to find some grounding in objective outcomes rather than simply 'consensus'. I think you hit upon the right firmament through your initial comparison to evidence-based medicine; the basis for urban planning decisions was, in the turn-of-the-20th-century sanitarian era, public health. The creation of infrastructure to prevent communicable disease was the impetus- i.e. water and sewer, movement to 'macadamized' roads that didn't allow pools of water to breed mosquitoes, elimination of horse transport and its attendant waste - and the broad notions that informed the city beautiful movement - access to light, air, trees, etc.

Of course, their evidence was at times scant or a vague understanding. The vague notions of pining for detached houses, light, and air certainly took a firm hold, aided with auto transport.

But the methodology for studying human behavior and the effects of multiple variables across large or small populations certainly exists, and human health objectives are certainly quantifiable, although it's easier to assail a particular measurement for - say, happiness - versus death from atherosclerosis.

But there has been a growing body of research over the last 15 years measuring the public health impact of urban form, neighborhood characteristics etc. This evidence is most robust in the realm of physical activity and land use/transportation, but it extends into other objectives as well. There is no need for the construct to be limited to typical 'medicine' ailments - the same epidemiological tools can be used to study any outcome and set of variables over a large population.

Ironically, planning seems to have less difficulty subjecting itself to the will of elected officials than it does with hewing to a set of public health-oriented objective goals. Is this because it obviates risk? No planner is required to stamp a small area plan to say that it will create x economic growth, preserve x cultural treasures, reduce incidence of depression and childhood asthma among the population living in the area, etc. - with risk of losing their 'license to plan' if it doesn't perform as expected.

But avoiding that responsibility means that planners will never garner the respect they often crave and frequently deserve - and plans will remain shelved or eviscerated by the city council du jour when it serves an immediate need.

If the field of planning could adhere to a set of core goals and utilize the extant tools of public health to garner funding for sufficient research to begin to define the absolute risk reduction across several outcomes of a 6 foot sidewalk versus a 3 foot sidewalk, it could achieve a stronger, more respected voice. Whether it is willing to go down that path with its attendant responsibilities is another question.

Gary Kueber, MD, MPH, MRP
Endangered Durham
Scientific Properties

Thank you

Great article! I think that we are headed there, slowly but surely. A new generation of quants who support the ideas of economists such as Stiglitz paired with the methodologies and community engagement of Jacobs is entering the urban planning fray, and is not satisfied with the "hand wave-y" solutions of the past--we want to see data tested empirically and not just nice sounding theories. Thanks to professionals such as yourself, jurisdictions are learning that there are quantitative methods at all, and that data previously unavailable can now be acquired with unprecedented ease and analyzed with powerful GIS, regression, database and econometric tools.

You are on the front line of what I expect will become a movement that brings hope and inspiration to current leadership in a time when it is sorely needed. Thanks for your work.

QBL Real Estate
Real Estate, Culture and Economics


Prepare for the AICP* Exam

Join the thousands of students who have utilized the Planetizen AICP* Exam Preparation Class to prepare for the American Planning Association's AICP* exam.
Starting at $245

Essential Readings in Urban Planning

Planning on taking the AICP* Exam? Register for Planetizen's AICP * Exam Preparation Course to save $25.
Grids and Guides Notepad Set

Spark Your Creative Thinking

Grids and Guides Notepad Set
Rome grey gold tie

Tie one on to celebrate your city!

Choose from over 20 styles imprinted with detailed city or transit maps.