What Can Planners Learn from Online Reviews? Examining Reviews of Los Angeles Rail Stations

Planners have long been eager to more effectively hear citizens' voices. My recent article in the Journal of Planning Education & Research demonstrates the potential of online review websites or social media as a platform that effectively connects planners and citizens. In this article, I explored the online reviews of Los Angeles County Metropolitan Transportation Authority (Metro) rail stations to examine how people evaluate rail stations. I particularly focused on how the user evaluation of rail stations vary by neighborhood type and over time.

The primary data source used for the study was 833 reviews of Metro rail stations posted on Yelp.com between 2007 and 2015. In addition to conducting a descriptive analysis of station ratings (one to five stars, from least to most favorable), I analyzed the contents of narrative reviews to identify the users' evaluation criteria. Major evaluation criteria were identified based on the stations' average percentage of reviews that mentioned each factor (e.g., cleanliness, parking). To differentiate users' feelings about a factor's importance from their satisfaction with that factor, I separated compliments and complaints for each factor.

Interestingly, there were not many extremely negative reviews. The average percentage of one-star reviews among final samples was found to be about 10%, compared to 16% in total Yelp reviews. The content analysis results also show that, on average, people tended to leave more compliments than complaints about Metro rail stations. These findings are somewhat surprising, given the earlier findings on or our stereotypes regarding L.A. public transit systems in general. However, this discrepancy may be due to people's different perceptions of rail versus bus transit, or the higher average socioeconomic status of rail users compared to bus riders.

The top five evaluation criteria included cleanliness, followed by surrounding neighborhoods, station location, crime-related safety, and parking (see table above). The most prevalent complaints concerned crime-related safety and the social environment. This is in line with the existing evidence on the fear of crime at Los Angeles transit stations. Other common complaints were parking, cleanliness, and the amenities of surrounding neighborhoods. Some evaluation criteria and complaint factors, such as parking and the social environment, reflect the context of the study area, although in previous literature these factors are rarely discussed.

Noteworthy are the neighborhood disparities in transit station quality. On average, stations located in low-income neighborhoods received lower ratings than those in middle-to-high-income areas, and these patterns did not change over time. The results of the content analysis indicate that the lower ratings for low-income area stations were not due to problems that are unique to these stations. Safety and security-related factors and cleanliness, for example, were among the most common complaints about stations in both low-income and middle-to-high-income neighborhoods. However, the percentage of reviews containing complaints was, on average, much higher for stations in low-income areas. The only remarkable difference was the relatively high percentage of reviews that complained about parking for middle-to-high-income area stations.

As for the temporal variations, a station's average rating and the number of reviews significantly increased over time for both types of neighborhoods. However, I found little temporal variation in the major evaluation criteria or complaint factors.

Some may argue that online review data suffers from sampling bias and therefore my findings do not necessarily capture the opinions of general transit riders. What if Yelpers with higher socioeconomic status rate transit stations differently than actual transit riders, for instance? To address these concerns, additional analyses were conducted based on the reviewers' sociodemographic information inferred from their narrative reviews about other businesses and profile photos (methodology adopted by Davis et al., 2019). Analyses by reviewers' socioeconomic characteristics suggest that the sampling bias in Yelp data can somewhat downwardly bias the overall ratings of stations. However, low-income area stations tended to receive lower ratings compared with middle-to-high-income area stations even after adjusting for reviewers' socioeconomic characteristics. Major evaluation criteria and complaint factors are also less likely to be affected by sampling bias.

My research provides valuable lessons for planners by identifying the station attributes that require special attention to improve metro users’ experiences and increase transit ridership. For example, station location was among the most important evaluation criteria. This implies the need for careful consideration of users' demands and convenience when selecting new station locations. In addition, the complaints frequently mentioned by reviewers indicate areas for improvement, including safety and security, cleanliness, and parking. Considering that the average rating for stations in low-income areas was lower, additional efforts should be made to reduce spatial disparities in these aspects of transit station quality. For stations in middle-to-high-income areas, providing sufficient parking in convenient locations should be prioritized to satisfy users and attract more choice riders.

My findings also urge planners to consider how to more effectively utilize social media information. Gathering user-generated content from social media is much cheaper and less labor intensive than conducting traditional user satisfaction surveys. Moreover, because people leave online reviews and share posts regarding various subjects and planning facilities, the potential for social media content to be useful as a resource for planning research and practice is not limited to transportation planning. Therefore, planners should endeavor to build best practices for analyzing and using social media content.

References:

Davis, D. R., Dingel, J. I., Monras, J., and Morales, E. 2019. “How segregated is urban consumption?.” Journal of Political Economy, 127(4): 1684-1738.>