The Coming Urban Data Revolution

<p> Historically, data sources for urban planning have remained relatively stable. Planners relied on a collection of well-known government-produced datasets to do their work, including statistics and geographic layers from federal, state and local sources. Produced by regulatory processes or occasional surveys, the strengths and limitations of these sources are well known to planners and many citizens. However all this is beginning to change. Not only has the U.S. Census Bureau&#39;s American Community Survey introduced a bewildering variety of data products, all with margins of error, three interrelated categories of new data are growing rapidly: crowdsourced, private, and &quot;big&quot; data. </p>

September 1, 2011, 7:44 AM PDT

By Robert Goodspeed @rgoodspeed


Historically, data sources for urban planning have remained relatively stable. Planners relied on a collection of well-known government-produced datasets to do their work, including statistics and geographic layers from federal, state and local sources. Produced by regulatory processes or occasional surveys, the strengths and limitations of these sources are well known to planners and many citizens. However all this is beginning to change. Not only has the U.S. Census Bureau's American Community Survey introduced a bewildering variety of data products, all with margins of error, three interrelated categories of new data are growing rapidly: crowdsourced, private, and "big" data.

First, crowdsourcing projects like Open Street Map and tools like Foursquare have created new maps and place data. Although tapping these sources currently requires technical skills beyond most users' capabilities, it seems likely access will become easier. Although not created through systematic surveys, they can often fill in blanks with higher levels of detail on topics previously difficult to study, such as crises.

Second, private consultants are peddling data to planners, often from proprietary sources and methods. ESRI's business and demographic data are notable examples. The latter is the result of projections from U.S. Census data that cannot be independently calculated. At the American Planning Association (APA) conference in Boston, the marketing firm Buxton advertised their services in the expo. The representative I spoke to admitted their products were similar to other marketing firms which stitch together public and private sources using proprietary techniques for market analysis.

Finally, the much-discussed "big data" may finally be coming to planning. Big data refers to large datasets that usually describe people and places, and are generated from administrative systems, cell phone networks, or other sources. Often created or compiled by the private sector, these datasets require specialized software to analyze and visualize. Some scholars are confident this data will shed new light on existing problems. However, the data are also useful if your aspirations are more prosaic -- illustrating a trend or creating a thematic map.

As consumers of this new data, planners and other public-sector clients are in a position to set expectations for data providers, as well as shed new light on old problems with new sources of information. Three guidelines are below, and I welcome additional discussion in the comments.

Demand transparent, replicable sources and methodologies. If the data will inform a public decision, even indirectly, the public interest demands it come from known sources and methods. Too many private data sources are rife with variables derived from "proprietary methodologies," which public clients can and should demand explanations about. All too often, what is hidden inside the black box is no better than a qualitative guess.

Use crowdsourced data thoughtfully. Although people may trust private data too much, the reverse is true for crowdsourced data. It can often be more accurate and detailed than the "official" sources. Therefore they should be used, but with a special effort to compare them to "known" sources and explain the variations and biases discovered. If no advantage is discovered their use can be abandoned, but often these sources can be amazingly detailed and useful for planners.

Link "big data" sources with planning issues. Too often data wonks are enthralled with the new sources of data, spending hours making pretty maps or visualizations that are interesting but ultimately irrelevant to substantive policy or planning questions. Although there is a place for visual exploratory data analysis for complex data and good design to communicate clearly with stakeholders, planners can play a key roll in these discussions. Instead of dismissing these efforts, urban planners should engage with data analysts and designers, guiding them towards relevant questions and policy issues. The Columbia Spatial Information Design Lab's projects epitomize this approach, mining public data to tell a policy-relevant stories about issues like prison spending or industrial zoning in Manhattan.

What are your favorite new sources? What additional guidelines should planners consider?


Robert Goodspeed

Robert Goodspeed is an Assistant Professor of Urban Planning at the University of Michigan. He holds a PhD from the MIT Department of Urban Studies and Planning and previously worked for the Boston Metropolitan Area Planning Council.

Chicago Commute

Planning for Congestion Relief

The third and final installment of Planetizen's examination of the role of the planning profession in both perpetuating and solving traffic congestion.

May 12, 2022 - James Brasuell

Twin Cities

Minneapolis Housing Plan a Success—Not for the Reason You Think

Housing advocates praise the city’s move to eliminate single-family zoning by legalizing triplexes on single-family lots, but that isn’t why housing construction is growing.

May 13, 2022 - Reason

Single-Family Housing Construction

New White House Housing Initiative Includes Zoning Reform Incentives

The Biden administration this morning released a new program of actions intended to spur housing construction around the United States.

May 16, 2022 - The White House

Complete Street Vancouver

Study: Most of Vancouver Is a ‘15-Minute City’

A large majority of Vancouver residents can access a grocery store in 15 minutes or less by bicycle or on foot.

May 20 - Vancouver Sun

Montreal, Quebec

Urban Design, Transport, and Health

The Lancet medical journal published a series of articles that explore how to evaluate and guide urban planning decisions to create healthy and sustainable cities. Live long and prosper!

May 20 - The Lancet - Urban Design, Transport, and Health

MoGo Bikeshare Bikes

Detroit Bike Share Celebrates Five Years

In its five years of operation, Detroit’s MoGo bikeshare has added electric and adaptive bikes to its fleet of more than 600 bikes.

May 20 - MoGo

HUD’s 2022 Innovative Housing Showcase

HUD's Office of Policy Development and Research

Expanding HUD’s Eviction Protection Grant Program

HUD's Office of Policy Development and Research

New Updates on The Edge

HUD's Office of Policy Development and Research

Urban Design for Planners 1: Software Tools

This six-course series explores essential urban design concepts using open source software and equips planners with the tools they need to participate fully in the urban design process.

Hand Drawing Master Plans

This course aims to provide an introduction into Urban Design Sketching focused on how to hand draw master plans using a mix of colored markers.