Describing these phenomena as auditory hallucinations seems inadequate. The aridly objective, notionally academic term gives no indication of the distress associated with the voices or the behavioural consequences. The associated phenomena are closer to the core of what a patient might be experiencing and closer to what might be considered to be the priorities in terms of a therapeutic intervention.

Behavioral scientists have conducted hundreds of studies about the differences between powerful and powerless words and phrases. We are especially smitten with research led by Jonah Berger at the University of Pennsylvania and by our Stanford colleague Jennifer Aaker. We draw mostly on their work to generate five tips about the kind of talk that provokes people to act, persist, and develop imaginative solutions.

Firms that attempted to replicate GPT with in-house data often produced shambolic “knowledge engines” that were little better than toys. These AIs supplemented the standard large-language-model training set with emails, mission statements, patent applications, legal memoranda, and other exciting selections from the internal corporate syllabus. As the buzz percolated through middle management, executives at marketing, media, and health-care firms launched ambitious initiatives, sometimes openly telling staff that many employees would be laid off once the neural nets were working. But much of what was produced was vaporware: late, expensive, and barely functional. Many users felt that AI technology simply

wasn’t ready.

In December 2023, an alarmed AI engineer at Microsoft, Shane Jones, would discover the downstream consequences of those decisions.

As he played around with Copilot Designer, Microsoft’s image generator built on DALL-E 3, he was horrified by how quickly it spit out offensive and sexualized images with little prompting. Just adding the term “pro choice” into the prompt, Jones found, produced scenes of a demon eating an infant and what appeared to be a drill labeled “pro choice” being used to mutilate a baby. Just prompting the tool for a “car accident” and nothing else produced sexualized women next to violent car crashes, including one in lingerie kneeling by a totaled vehicle, CNBC subsequently found through its own testing.

Nitasha Tiku, the reporter who broke the story, also spoke to Emily Bender and Meg Mitchell, who had warned in their “Stochastic Parrots” paper about the problem of large language models fooling people into seeing real meaning and intent behind their generations. “We now have machines that can mindlessly generate words, but we haven’t learned how to stop imagining a mind behind them,” said Bender.

“I’m really concerned about what it means for people to increasingly be affected by the illusion,” when that illusion is becoming so good, Mitchell said.