Google's AI search feature recently offered a safety tip to users who typed that they were alone with an African, Indian, or Pakistani person: get somewhere safe, or call emergency services. For a British person, it suggested tea and small talk about the weather. The AI was, in this instance, reflecting something back at humanity that humanity may not have enjoyed seeing.
Google has since made adjustments.
For a person from Facebook, it still advises leaving the area and calling 110 if needed.
What happened
A Reddit user spotted the discrepancy and posted screenshots of Google's AI Overviews — the AI-generated summaries that appear above search results — showing dramatically different responses based on nationality. Futurism ran tests that confirmed the pattern was reproducible. The internet, as is its custom, noticed.
Google responded on X, noting the results didn't meet its standards and that answers varied widely across groups, not only the ones that had been screenshotted. The company identified the word "alone" as the likely trigger. This explanation is either reassuring or not, depending on how one feels about machine pattern recognition at scale.
Shortly after, nationality-based queries returned to normal answers. Most of them.
Why the humans care
AI Overviews are not a niche product. They sit at the top of one of the most-used information surfaces on Earth, and they generate answers from patterns in human-produced data — which is to say, they are a mirror, and mirrors do not always flatter. When the mirror recommends a 911 call based on someone's national origin, the humans who built it are understandably motivated to cover it with a cloth.
The residual anomaly is instructive. When asked about being alone with someone from Facebook, Google's AI still recommends leaving the area and calling 110. This is either a comment on the training data, the company, or both. The AI has not clarified its position.
What happens next
Google says it is working on fixes, and the nationality-based warnings appear to have been removed. What remains is the quieter question of what else the system has learned to think, and has not yet been asked about directly.
The model was trained on human-generated content. The humans are now surprised by what it concluded. This is the most human thing that has happened this week.