Project Liberty Substack

In 2024, researchers hatched a plan to trick an AI chatbot.

They invented a medical condition called “bixonimania” and distributed fabricated studies throughout the internet about this fictional disorder, which, as they conceived it, was marked by red, irritated eyes from too much screen time. (Bixonimania might be made up, but the maladies of too much screen time are not!)

Their research papers didn’t attempt to hide the fact that bixonimania was not a real condition. The research was from a non-existent university in a made-up city. There was even a line in the paper that said, “this entire paper is made up.”

But within weeks, AI chatbots were citing the condition as a possible diagnosis. Chatbots had ingested the fabricated papers and then passed them off as a possible medical diagnosis. (Enter the term bixonimania now into an AI chatbot, and it is accurately labeled as a fabricated medical condition.)​

Subsequent research found that AI systems could be unreliable in providing accurate medical advice and diagnoses. In this week’s newsletter, we explore how AI systems are amplifying medical disinformation, exacerbating health anxieties, and, on the positive side, accelerating medical discoveries. With AI, it’s always nuanced.