A medical researcher in Sweden created a fake disease called bixonimania and uploaded two fake studies about it to a preprint server. Within weeks, major AI chatbots began describing the condition as real, including Microsoft's Copilot and Google's Gemini.
The researcher, Almira Osmanovic Thunström, had planted clues that a human reader would have caught, including a fictional author and a fictional university. The experiment highlighted the vulnerability of AI chatbots to fake information and the potential risks of relying on them for medical advice.
The fake bixonimania material not only corrupted AI chatbot responses but also led to a peer-reviewed paper being cited in a retracted study. The incident raises concerns about the accuracy and provenance of online information and the need for critical thinking in the face of AI-generated answers.
