NEW YORK, September 26 — The shift is here, and it landed on the weather desk. A journalist put the same question to four of the biggest conversational AI systems on the market — ChatGPT from OpenAI, Gemini from Google, Claude from Anthropic, and Muse — asking each how severe the approaching weekend nor’easter would be. The comparison, framed around a single practical question, cuts straight to whether general-purpose chatbots can be trusted for time-sensitive severe-weather guidance.
What the test really exposes is a category error that plenty of users are still making. These systems are large language models accessed through a conversational interface, not dedicated meteorological services.
They generate plausible-sounding text from patterns in their training data. They do not run their own atmospheric simulations, and their outputs are not official forecasts.
The nor’easter, and who actually forecasts it
The storm in question is a nor’easter — a system along the US East Coast whose cyclonic circulation drives persistent northeasterly winds onshore, typically bringing heavy rain or snow, coastal flooding and beach erosion. The warning for the coming weekend is what prompted the comparison in the first place. Authoritative severe-weather guidance in the United States comes from the National Weather Service and its National Hurricane Center.
That is the benchmark, and it is not a chatbot. Hallucination is the known failure mode here.
A model can state incorrect information confidently, and that is precisely the wrong behaviour when someone is deciding whether to travel or whether a school closes. A related problem sits underneath it. Models without live data access may lean on stale training information, or on web retrieval of uncertain quality.
Why the answers stay vague
Watch what the chatbots actually do. They generally avoid firm numeric commitments unless grounded in a retrieved source, and they typically append caveats pointing users to official forecasts. That caution is sensible engineering.
It is also a signal that the systems themselves are not the source of record. The practical takeaway is procedural, not technological.
Treat a chatbot answer about an incoming storm as a starting point for understanding terminology and preparedness categories. Then verify against the National Weather Service and local emergency management before making travel, school or evacuation decisions. No forecast totals, wind speeds, snowfall ranges or landfall locations were established in the comparison, and no claim about which model performed best can be made from it.
That absence is itself the story. The trend underneath is that consumers are reaching for generative AI in high-stakes, time-critical moments where accuracy and sourcing are directly consequential — and the tools are not built for that job yet.
Watch this space. The gap between a confident-sounding answer and an official forecast is where the next round of AI product decisions will be made.
