AI promises to finally make public engagement meaningful. We put it to the test.

This piece is part of a collaborative series analyzing the 2026 surface transportation reauthorization process. Read more here. 

Imagine more than 13,000 public comments and 5.6 million words in a single federal docket. Where would you even start?

In September, 2025, The U.S. Department of Transportation finished collecting feedback for its Request for Information (RFI) on  Surface Transportation Reauthorization receiving over 13,000 comments. Responses varied in length, from single pages to documents over 50 pages long, and came in formats including text files, PDFs, Word files and scanned images.

If you’ve worked in planning, you probably know this feeling. During a comprehensive plan update or a zoning hearing, public comment periods often bring in hundreds or even thousands of responses. Fu et al. (2024) pointed out that reviewing all this public feedback by hand takes more time and resources than most agencies have. As a result, this kind of public feedback can only usually be skimmed rather than carefully reviewed. When feedback is not fully analyzed, public participation can start to feel like just another box to check instead of a meaningful process.

I am a master’s student in urban planning at Texas A&M University, and I've been asking a central question in my studies: Can artificial intelligence bridge this gap? Instead of replacing human judgment, can it be used to identify patterns across millions of words, highlight areas of disagreement and ensure minority voices are not lost?

To explore this question, our team took the RFI as an opportunity to test three AI tools. Using them, we analyzed the full RFI dataset to find key themes requested by citizens (Piece 1), then took a closer look at safety (Piece 2) and active transportation (Piece 3) in a policy context.

Recently, Planetizen published the findings from our analysis in a piece by Kristina Curtiss called "What does America want from surface transportation reauthorization?" which explains what we found in the 13,000+ public comments. 

In this article, I would like to share how we did it and show that even if you do not have a big research team or expensive software, your organization can start using these methods right away.

How we did it

For our analysis, we followed five key steps:

1. Collecting comments from Regulations.gov 
2. Extracting text from 723 attachments 
3. Consolidating all the data into one master CSV file
4. Running analyses with several AI tools 
5. Summarizing the results into themes and insights

The data

Out of over 13,000 comments, 592 institutions included 723 attachments. Most attachments were PDFs (88%), with Word documents accounting for 9% and other formats such as RTF and image files accounting for the remaining 3%. The amount of content in each comment and document varied significantly. For example, comment #2667 had 8,600 signatories and 2,064,312 words across five PDF files, but most of it was a form-letter campaign using a similar template with different names and addresses.

Collecting and preparing data

Luckily, Regulations.gov has a Bulk Data Download service. We used this service to download a CSV file with the comment text and metadata. However, since the CSV file listed only URLs for the attachments, we wrote a Python script that used the Regulations.gov API to automatically download all 723 files.

Next, we extracted text from the attachments. For PDFs and Word documents, we used libraries like pdfplumber and python-docx. For scanned image files, we used Tesseract, which is an open-source OCR tool. We matched the extracted text to the original comments and combined them into a single master CSV file, with each row representing a comment and its attachment text. For the large attachment in #2667, we wrote "Refer to attached files: …" in the attachment text column and kept the original PDFs. Due to the AI token limit, we split the master CSV into six files, each about 2 MB. Depending on the AI’s token capacity, splitting the file may or may not be needed.

The most time-consuming part was preparing the data, which meant writing code and cleaning up the files. This step is the most important because the quality of the data affects the quality of the AI analysis.

Our AI toolkit

One important thing to note is that, because we used a University Enterprise account, our input data was not used to train the AI. Even though the public comments in this study are already available for everyone, this approach is important because it can be scaled up. If you want to use the same workflow for confidential planning documents or sensitive feedback, it’s essential to use an environment where your data won’t be used for AI training.

Prompting strategy

If you just ask the AI to "Summarize these comments," you’ll only get surface-level results. Instead, we gave AI tools this specific role and structured task:

I am conducting a qualitative analysis of public comments for a federal RFI. I have provided 13,000+ comments (via CSV) and detailed technical attachments for a specific large-scale entry.

Task: Act as a Policy Analyst and synthesize all provided information into a comprehensive research report based on the following requirements...

Five key tasks are outlined below the prompt:

1. Primary Themes: Identify the top 5-7 recurring topics discussed across all comments.

2. Key Stakeholder Perspectives: Briefly summarize how different groups differ in their primary concerns.

3. Emerging Keywords: List unique or technical terms that appear frequently within the dataset.

4. Gap Analysis: Are there any obvious tensions or conflicting opinions between different groups of commenters?

5. Actionable Insights: Based on the entire dataset, what are the 3-5 most critical points the agency should address?

What we learned

Each tool has its strengths and weaknesses

We used the same dataset and the same prompt, but the three models interpreted the comments in different ways. All three did well with source grounding, using the original CSV files or PDFs to back up their findings. However, there were some differences in how they handled certain areas.

• ChatGPT 4.1 was best at a balanced summary. It provided a great overview across all seven themes, thus making it the most useful for quickly catching the big picture. It also covered technology topics, like V2X, digital project delivery and autonomous driving, which other models tended to overlook, though it fell short in specific figures and data points.
• NotebookLM was great at quickly organizing information and spotting niche issues. It clearly defined and summarized specialized policy terms, such as NEPA, VRU, OFD, PFAS and SS4A. It also raised issues that other models didn’t address, such as PFAS liability, suballocation and conflicts over truck size and weight rules. However, it gave the least detailed supporting evidence or numerical data.
• Claude Opus 4.5 stood out for its precise and in-depth analysis. It cited specific figures, such as the $140B state of good repair backlog, 80-97% reduction in collisions and 86% bipartisan support. The analysis also identified six points of tension and suggested a resolution for each. However, its results were the longest, which made it harder to quickly find the main points.

What actually drove performance

• Preparing your data is more important than which model you choose. The better the groundwork, the better the results were for all three models. If your data is messy, your results will be messy, no matter which model you use.
• Using multiple models helps reduce blind spots. If we had only used NotebookLM, we would have missed the $140 billion backlog in transit operating funds. We noticed these gaps when we compared the results from each model.
• AI cannot do it all. AI worked well with text-based documents, but attachments with lots of charts and diagrams still needed a human to review them. When different models gave conflicting results, human judgment was also necessary. For example, Claude saw active transportation conflicts as a deep ideological divide, while other models were less pessimistic. AI can speed up analysis, but knowing its limits was a key part of our approach.

How local agencies can apply this

Even if your organization doesn’t deal with federal data, this workflow can still be effective, because it’s about structure, not size. Whether you have 300 public comments on a comprehensive plan or 500 comments after a zoning change hearing, you can use the same process, and the upfront costs are also surprisingly low.

There are almost no cost barriers. Many AI tools are available for free. Even Tesseract OCR, which we used for data cleaning, is open source. You can get started without an enterprise account. Just remember to consider data security separately.

You also don’t need advanced coding skills. AI helped us to write most of the Python code we used. A basic understanding of Python is enough. After the initial setting, you can start using AI for analysis in just a few minutes — just make sure not to enter API keys or personal information into the AI.

You can start small. For smaller organizations, simply uploading data to the AI and providing clear prompts is enough to generate useful insights. As your dataset grows, you can scale up by using the Retrieval-Augmented Generation method, where you upload documents and have the AI reference only those files. NotebookLM uses this RAG method, which helps minimize AI hallucinations.

Practical tips

• Make data security your top priority. The main benefit of an Enterprise account is that your data isn’t used for AI training. If you cannot guarantee this, it’s safest to use only public materials, like public comments, hearing transcripts and past master plan reports. Be sure to mask any personal information that isn’t anonymous.
• Don’t be afraid to split your files. If you upload too much data at a time, you’ll hit the AI’s token limit. Check how much text each tool can handle and split your files as needed. For long documents, upload them in chapters or sections. This helps the AI focus on each topic and give more precise results.
• Most AI tools provide a feature for managing data by project. For example, ChatGPT has Projects, Claude has Workspace and Gemini has Gems. By uploading your split files into a single project, you can keep your analysis organized and maintain context, even as your conversation grows longer.
• Use the same prompts and check them across different tools. A prompt like "Summarize these comments" only gives you a basic overview. It’s better to assign a role, like "Act as a policy analyst," and give a clear task. As we found, one tool might spot an issue that another might miss. The more tools you use, the fewer blind spots you’ll have.
• Finally, treat AI outputs as drafts. When results from different models don’t match, it’s essential to return to the original data and check it yourself.

Limitations to keep in mind

This methodology speeds up public comment analysis, but it does have several limitations.

• There are limits to processing visual materials. While AI worked well with text-based documents, it still cannot fully understand photos and map images in attachments. In our analysis, we needed to review these kinds of materials by hand.
• There’s also the issue of the digital divide and sample bias. Public comments often reflect the voices of people with good online access and who are comfortable with text-based communication. No matter how advanced the AI, it cannot analyze voices that are not in its data, such as those of the elderly, low-income individuals or people whose first language is not English. This limitation is not a flaw in the AI but in the public comment system itself, and it is important to keep this in mind when reviewing the results.
• Minority opinions can get lost. When sorting thousands of comments into themes, less common but important insights might be missed. Since AI often focuses on the most frequent themes, you may need to write prompts that specifically look for important minority opinions.
• There is also a risk of hallucination. AI can sometimes make up information. Using the RAG approach and source grounding helps reduce this risk, but errors can still occur. Always check AI outputs against the original data.

Even with its limitations, AI has the potential to help organizations handle thousands of citizen feedback items. While it may not be a perfect solution, careful use of AI could ensure more voices actually shape the decisions that affect people’s lives.

Yoonjeong Kang is a Master of Urban Planning student at Texas A&M University. Her research focuses on AI applications in planning practice and transportation policy analysis.