Open Data: Coming to a City Near You?

City data catalogs are fast moving from the exception to the norm for large U.S. cities.

Washington, DC's Data Catalog, spearheaded by former CTO Vivek Kundra, was an early leader. The site combines hundreds of static government-created datasets from across DC government with administrative feeds like the city's 311 system. Their site emphasizes providing data in multiple formats, including where possible formats that don't require proprietary software. Kundra's selection as the nation's first Chief Information Officer, and launch of the federal government's Data.gov has elevated the principle among the federal government's vast datasets. DC's two "apps" contests sought to encourage creative uses of the data made available, and some of which are available at the DC App Store.

Beyond DC, many big cities have recently launched or are planning open data catalogs of their own.

Last week, New York City launched their Big Apps project, an app competition which coincided with the initial public release of their NYC Data Mine website. Although my colleague Anthony Townsend has some valid criticisms of the effort thus far, it still initiates the first steps for an open data program in New York City. Relatively new projects abound in other cities also. Boston recently launched a Data Hub that allows download of a handful of spatial datasets, including reports from the Mayor's Hotline. Vancouver is developing an Open Data Catalogue, and DataSF has a variety of datasets. Toronto seems poised to follow with their toronto.ca/open initiative planning to release initial data this winter. Outside of city government, innovative "indicators" projects are making data available in new ways, often in raw formats. (One example is the Greater New Orleans Community Data Center) The regional planning agency I worked for previously operated a regional government data portal, the MetroBostonDataCommon, which allowed for shapefile downloads.

Plenty of skepticism surrounds these initiatives, however. The DC Stumble Safely app, which combines liquor license data with crime reports to guide bar-goers safely home, is pointed to as a successful example of realizing value of the data. One concern, mentioned in Nate Berg's report from the Open Cities conference, regards the equity implications of making data available in way that is only useful to those who already have expensive GIS software or pricey gadgets like the iPhone. More generally, in a world with acute social and economic problems, it seems like an unnecessary use of government funds. The primary answer to the cost concern is that making data public isn't nearly as expensive as many think. Some innovative small cities, like Nanaimo, British Columbia are proving that. The datasets are often already created, and the costs of hosting data minimal. However, is there a greater goal than simply encouraging transparency - will it actually improve how we understand the city? In a recent post on my personal blog, "Does Data Matter in Urban Policy?," I argue data availability is connected to a larger political spirit that explicitly values and looks to data as a source of a common framework for policy creation and debate. Where ideological agreement eludes us, data is championed as a possible shared basis for understanding.

What other city data portals have I missed? Which approach should cities take, spending funds on lavish mapping applications, or slapping up .csv's? Is XML good enough, or (as developer Dan Knauss suggested to me recently) do governments need queryable databases for dynamic data?