The Coming Urban Data Revolution

Historically, data sources for urban planning have remained relatively stable. Planners relied on a collection of well-known government-produced datasets to do their work, including statistics and geographic layers from federal, state and local sources. Produced by regulatory processes or occasional surveys, the strengths and limitations of these sources are well known to planners and many citizens. However all this is beginning to change. Not only has the U.S. Census Bureau's American Community Survey introduced a bewildering variety of data products, all with margins of error, three interrelated categories of new data are growing rapidly: crowdsourced, private, and "big" data.

3 minute read

September 1, 2011, 7:44 AM PDT

By Robert Goodspeed @rgoodspeed


Historically, data sources for urban planning have remained relatively stable. Planners relied on a collection of well-known government-produced datasets to do their work, including statistics and geographic layers from federal, state and local sources. Produced by regulatory processes or occasional surveys, the strengths and limitations of these sources are well known to planners and many citizens. However all this is beginning to change. Not only has the U.S. Census Bureau's American Community Survey introduced a bewildering variety of data products, all with margins of error, three interrelated categories of new data are growing rapidly: crowdsourced, private, and "big" data.

First, crowdsourcing projects like Open Street Map and tools like Foursquare have created new maps and place data. Although tapping these sources currently requires technical skills beyond most users' capabilities, it seems likely access will become easier. Although not created through systematic surveys, they can often fill in blanks with higher levels of detail on topics previously difficult to study, such as crises.

Second, private consultants are peddling data to planners, often from proprietary sources and methods. ESRI's business and demographic data are notable examples. The latter is the result of projections from U.S. Census data that cannot be independently calculated. At the American Planning Association (APA) conference in Boston, the marketing firm Buxton advertised their services in the expo. The representative I spoke to admitted their products were similar to other marketing firms which stitch together public and private sources using proprietary techniques for market analysis.

Finally, the much-discussed "big data" may finally be coming to planning. Big data refers to large datasets that usually describe people and places, and are generated from administrative systems, cell phone networks, or other sources. Often created or compiled by the private sector, these datasets require specialized software to analyze and visualize. Some scholars are confident this data will shed new light on existing problems. However, the data are also useful if your aspirations are more prosaic -- illustrating a trend or creating a thematic map.

As consumers of this new data, planners and other public-sector clients are in a position to set expectations for data providers, as well as shed new light on old problems with new sources of information. Three guidelines are below, and I welcome additional discussion in the comments.

Demand transparent, replicable sources and methodologies. If the data will inform a public decision, even indirectly, the public interest demands it come from known sources and methods. Too many private data sources are rife with variables derived from "proprietary methodologies," which public clients can and should demand explanations about. All too often, what is hidden inside the black box is no better than a qualitative guess.

Use crowdsourced data thoughtfully. Although people may trust private data too much, the reverse is true for crowdsourced data. It can often be more accurate and detailed than the "official" sources. Therefore they should be used, but with a special effort to compare them to "known" sources and explain the variations and biases discovered. If no advantage is discovered their use can be abandoned, but often these sources can be amazingly detailed and useful for planners.

Link "big data" sources with planning issues. Too often data wonks are enthralled with the new sources of data, spending hours making pretty maps or visualizations that are interesting but ultimately irrelevant to substantive policy or planning questions. Although there is a place for visual exploratory data analysis for complex data and good design to communicate clearly with stakeholders, planners can play a key roll in these discussions. Instead of dismissing these efforts, urban planners should engage with data analysts and designers, guiding them towards relevant questions and policy issues. The Columbia Spatial Information Design Lab's projects epitomize this approach, mining public data to tell a policy-relevant stories about issues like prison spending or industrial zoning in Manhattan.

What are your favorite new sources? What additional guidelines should planners consider?


Robert Goodspeed

Robert Goodspeed is an Assistant Professor of Urban Planning at the University of Michigan. He holds a PhD from the MIT Department of Urban Studies and Planning and previously worked for the Boston Metropolitan Area Planning Council. See his academic website for more on his teaching and research.

Aerial view of homes on beach in Maui, Hawaii

Hawaii Passes First Legislation Regulating Short-Term Rentals Statewide

The new law will give counties the power to limit number or short-term rentals and convert existing short-term rental units back into long-term residential housing.

May 13, 2024 - USA Today

Entrance to a drive-through car wash at night with green 'Enter' sign.

Ohio Towns Move to Ban New Car Washes

City officials in northeast Ohio are putting limits on how many car wash facilities can open in their towns.

May 16, 2024 - News 5 Cleveland

Aerial view of Oceanwide Plaza skyscrapers covered with graffiti tags.

LA’s Abandoned Towers Loom as a “$1.2 Billion Ruin of Global Capital”

Oceanwide Plaza, shuttered mid-construction after its developer filed for bankruptcy, has stood vacant on prime Los Angeles real estate since 2019.

6 hours ago - The Architect's Newspaper

Aerial view of Oceanwide Plaza skyscrapers covered with graffiti tags.

LA’s Abandoned Towers Loom as a “$1.2 Billion Ruin of Global Capital”

Oceanwide Plaza, shuttered mid-construction after its developer filed for bankruptcy, has stood vacant on prime Los Angeles real estate since 2019.

6 hours ago - The Architect's Newspaper

Close-up of back windshield of Waymo autonomous car.

NHTSA Launches Investigations Into Two More Self-Driving Automakers

The federal agency is responding to reports of crashes and erratic behavior from Waymo and Zoox vehicles.

7 hours ago - Smart Cities Dive

Texas Townhomes

Austin Reduces Minimum Lot Size

Smaller lot sizes for single-family homes can make construction more affordable and homeownership available to more households.

May 21 - The Texas Tribune

News from HUD User

HUD's Office of Policy Development and Research

Call for Speakers

Mpact Transit + Community

New Updates on PD&R Edge

HUD's Office of Policy Development and Research

Write for Planetizen

Urban Design for Planners 1: Software Tools

This six-course series explores essential urban design concepts using open source software and equips planners with the tools they need to participate fully in the urban design process.

Planning for Universal Design

Learn the tools for implementing Universal Design in planning regulations.