5 ways to hallucination-proof your AI output

This article is part of an ongoing column on AI and planning by urban planning professor and AI expert, Tom Sanchez. Learn more about Tom and read more installments of his column. 

Last month I wrote about prompt engineering, the skill of asking AI better questions. In some follow-up conversations, one question was: once you get a good answer, how do you know it is actually right? That question points to the second half of a skill set that is just as important and can take significant effort. Prompting is how you get useful output from AI. Verification is how you determine whether to trust it and whether you can use it. One without the other is only half of an effective process.

Here are two examples of why this matters now. 

Late last year, Deloitte agreed to partially refund the Australian government after a AU$440,000 commissioned report was found to contain references to nonexistent academic works and a fabricated quotation attributed to a real federal court judgment. Deloitte acknowledged that generative AI had been used in preparing the report (AP News). In 2023, two New York attorneys were sanctioned in Mata v. Avianca after filing a legal brief containing six case citations that ChatGPT had invented (The New York Times). And in my own work over the past year reviewing AI-assisted manuscripts and reports and experimenting with these tools, I have frequently encountered references that looked completely legitimate (real journal names, plausible author teams, realistic volume and page numbers) but that do not exist anywhere. Some were subtle: a real author along with a title they never wrote, or a real article assigned to the wrong journal and year.

If this can happen to global consulting firms and practicing attorneys, it can happen to planners. And the documents produced (staff reports, comprehensive plan chapters, grant applications, and environmental review materials) are public records with legal consequences. An invented reference in a staff report is not just embarrassing. It can undermine the record supporting a council decision, lead to an appeal, and chip away at something planners cannot afford to lose: public trust. I made that argument early in this column series, and the arrival of AI in daily practice has only raised the stakes.

Why fluency can be tricky

To verify AI output, it helps to understand why these systems get things wrong. Large language models do not look facts up in a database. They generate text by predicting what words are most likely to come next, based on patterns learned from enormous amounts of training data. Much of the time, the most plausible continuation is appropriate or correct. The model nearly always produces something plausible, whether or not it has solid information to back it up. When the AI creates "facts" that are not real, we call the output a hallucination — though I find "made-up stuff" more descriptive.

The trap for planners is that some of the cues we normally use to judge written work no longer apply; in human writing, polish, specificity and confidence usually point to competence. A memo with precise figures, properly formatted citations, and an assured tone typically reflects careful work. AI output has all of those surface qualities, whether it is right or wrong. A Stanford study of leading AI legal research tools, products specifically marketed for reliability, found they produced incorrect or ungrounded answers between 17% and 33% of the time (Magesh et al., 2024). The point is not that AI is useless. Fluency tells you little, so verification has to do the work that our instincts used to do.

5 ways AI output goes wrong

The burden of catching errors comes down to knowing what to look for. Fortunately, these errors are not random. Across planning-related tasks, there are five recognizable types, each with its own warning signs. Learning to recognize the categories helps turn verification from a vague worry into a habit: instead of asking "is any of this wrong," learn to ask five specific questions that will help to catch most problems.

1. Fabricated sources. The output cites a study, report or legal case that does not exist, or attaches real authors to imaginary titles. This is a dangerous error because fabricated references look exactly like real ones.
2. Plausible but wrong facts. The date, number or attribution is close but incorrect. The output says a policy started in 2019, when it actually started in 2021, or attributes a policy to the wrong agency. These errors make it through casual review precisely because they are plausible.
3. Outdated information. Models are trained on data with a cutoff date, and even tools with web access do not always check for updates. Zoning codes get amended, thresholds change and case law evolves. An answer that was accurate two years ago may be wrong today.
4. Quantitative slips. LLMs are pattern generators, not calculators. Asked to summarize a table, compute percentage change or scale an estimate, they may produce numbers that are subtly off or internally inconsistent.
5. Misapplied context. The answer is generally true but wrong for your jurisdiction. AI trained mostly on widely available material tends to describe common practice, which may not match your state's enabling legislation or your city's code. Accessory dwelling unit rules are a good example: the correct answer in California is the wrong answer in Texas.

What these five types of errors share is that none of them necessarily look like errors. They are generated in the same fluent, confident prose as everything else, which is why a habit of verification is better than a sense of when something feels off. Turn the categories into five quick questions: Do the sources exist? Are the specifics right? Is it current? Do the numbers hold up? Does it fit your jurisdiction? Those five checks catch the large majority of what goes wrong.

Match the effort to the stakes

Nobody has time to fully verify every AI response, and nobody wants to. A sensible principle is proportionality: the level of verification should match the consequences of being wrong. I think of it as three tiers.

At the low-stakes tier (brainstorming, outlining, exploring a topic you plan to research properly anyway), a light review may be fine. The output is raw material, not a product, and errors get filtered out downstream.

At the middle tier (internal memos, presentation content, background research that colleagues will rely on) verify the most visible elements: every citation, every statistic, every claim that someone might repeat in a meeting. You do not need to audit every sentence, but nothing checkable should go forward unchecked.

At the high-stakes tier (staff reports, plan language, grant narratives, anything entering the public record or supporting a decision), verify everything. A rule of thumb: Treat AI output headed for the record the way you would treat a draft from a new intern whose work you must sign. You wouldn’t forward it on without review, however polished it looks.

Five verification habits worth trying

Knowing the failure types is half the battle; the other half is building verification into your routine. The five habits below cover many of the possible issues, require no technical expertise and target the errors most likely to end up in a staff report or public document.

1. Trace key facts to a primary source. For any statistic or factual claim that matters, find where it actually comes from: the census table, the statute, the adopted plan. Tools that search the web and show their sources make this much faster, but a listed source only helps if you open it and confirm it says what the AI claims. Asking the model "what is your source?" is not verification; models can fabricate justifications as fluently as they fabricate facts.
2. Check the math. Recompute a sample of any calculations. Check that percentages sum, that per-capita figures pass the smell test, and that units and orders of magnitude make sense. If a number would surprise you when it comes from a colleague, it should trigger a check from AI.
3. Get a second opinion. Run important questions past another model, or ask the same model to critique its own answer and look for weak points. Disagreement between passes does not tell you which answer is right, but it may flag where to focus your human review.
4. Apply your local knowledge. This is the check only you can perform. Does the output match what you know about your community, your code, your politics, your geography? Planners' expertise and contextual knowledge is precisely what AI lacks. When output conflicts with what you know to be true, trust your knowledge and intuition and dig deeper, just to be safe.
5. Click every citation. Do not skim the reference list and judge whether it looks right. Confirm each source exists: search the title, open the link. If you cannot locate a source easily, assume it is fabricated until proven otherwise. This can take time and effort you thought you were saving by using an AI tool.

One more habit ties all of these together: keep a simple record of what you verified. A column in a spreadsheet or a note in the file ("all citations confirmed 7/10/26, data checked against ACS 2024") takes a minute and creates accountability.

From personal habit to agency practice

Everything above describes individual skills, and individual adoption is where most planners are right now. But agencies eventually need to turn these habits into policy. In reviewing local government AI policies over the past year, I have noticed that several require "human review" of AI-generated content, but few define what that review entails. A requirement that is not operationalized is not going to protect against any of the potential issues mentioned here.

Planning agencies can do better with three concrete moves: define verification standards by document type (using the tiers above), name a responsible reviewer for anything AI-assisted that enters the public record and document the verification the same way we document other quality control. This is what "human in the loop" means in practice. Otherwise the loop is just a rubber stamp.

The skill that stays human

It is tempting to see verification as a "tax" we pay for using AI. I would frame it differently. AI could shift the composition of professional work away from producing first drafts and toward evaluating, correcting and taking responsibility for them. That kind of judgment (weighing evidence, checking claims against what you know about the community, deciding what belongs in the public record) has never been a side task. It is what planners do. It is planning judgment, applied to a new kind of input. Using these tools will not lead to generating more content, but they should lead to more trustworthy outputs, AI-generated or not.

As always, I would love to hear how you are handling this in your own practice, including the verification steps your agency has adopted and any errors you have caught. You can comment in the section below and, if you have a question you would like answered, please email [email protected] with "AI question for Tom Sanchez" in the subject line. Let's learn together how to put AI to work in ways that truly serve our communities.

Reference

Magesh, Varun, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning, and Daniel E. Ho. "Hallucination‐free? Assessing the reliability of leading AI legal research tools." Journal of Empirical Legal Studies 22, no. 2 (2025): 216-242.