Cal Fire Chatbot Fails to Answer Basic Questions

California’s state fire protection agency has launched an AI chatbot in an effort to improve efficiency — and the results are troubling, reports Malena Carollo in The Markup. “The chatbot uses the Cal Fire website and the agency’s ReadyForWildfire.org to generate answers. It can tell users about topics such as active wildfires, the agency, fire preparedness tips and Cal Fire’s programs.”

CalMatters found that the chatbot, which is meant to offer “critical fire prevention resources and near-real-time emergency information,” “fails to accurately describe the containment of a given wildfire, doesn’t reliably provide information such as a list for evacuation supplies and can’t tell users about evacuation orders.”

The chatbot had trouble answering questions about fire preparedness depending on how questions were phrased. “And while the chatbot didn’t generate incorrect answers in any of the queries CalMatters made, it doesn’t always pull the most up-to-date information” — for example, showing fire containment information that was six days out of date. The chatbot is also unable to provide information about evacuation orders.

Experts say testing of the bot should have happened long before it was launched for public use. “Agencies interested in using chatbots should identify the questions the public is likely to ask the AI tool ahead of time, ensure those are representative of the expected population the agency serves and refine the chatbot by having members of the public pilot the system to ensure it provides the kind of information they seek.”