This is known as the “alignment” problem. It’s discussed at length in other excellent books, like Superintelligence, Stuart Russell’s Human Compatible, and Brian Christian’s The Alignment Problem, so I won’t go into it in depth here.

In reality, the analogies to intelligence are once again anthropomorphizing and exaggerating the capabilities of the technology. While Hinton and other deep learning absolutists predicted that the shortfalls of neural networks compared with humans would go away at sufficient scale, the challenges have in fact persisted and, by many accounts, only gotten worse.

During my conversation with Brockman and Sutskever, I had referenced one of them: the carbon footprint of training such models. In June 2019, Emma Strubell, a PhD candidate at the University of Massachusetts Amherst, had been the first to coauthor a paper showing that the footprint for developing large language models was growing at a startling rate. Where neural networks could once be trained on powerful laptops, their new scale meant their training was beginning to require data centers drawing significant amounts of energy from carbon-based sources. In the paper, Strubell estimated that training the version of the Transformer that Google used in its search for just a single cycle—in other words, feeding it some data and letting it compute a statistical model of that data—could consume roughly 1,500 kilowatt hours of energy.

In her 2019 talk at NeurIPS, during the Queer in AI workshop, Ria Kalluri, an AI researcher at Stanford, proposed an incisive alternative to the question of how to ensure AI does “good.” Goodness, benefit to humanity—these terms will always be in the eye of the beholder. Rather, we should ask how AI shifts power: Does it consolidate or redistribute that power? To put it in the frame of this book, does it continue to fortify the empire, or does it begin to wrest us back toward democracy?

Finally, to redistribute power along our third axis, influence, we need broad-based education. The antidote to the mysticism and mirage of AI hype is to teach people about how AI works, about its strengths and shortcomings, about the systems that shape its development, about the worldviews and fallibility of the people and companies developing these technologies. As Joseph Weizenbaum, MIT professor and inventor of the ELIZA chatbot, said in the 1960s, “Once a particular program is unmasked, once its inner workings are explained in language sufficiently plain to induce understanding, its magic crumbles away.” I hope this book is just one offering to help induce understanding. It builds on the work of the many scholars, journalists, activists, and educators before me who have dedicated themselves to public education. May it be a new ground upon which many more after will rise up and build.