My conversation with Brockman and Sutskever continued on in circles until we ran out the clock after forty-five minutes. I tried with little success to get more concrete details on what exactly they were trying to build—which by nature, they explained, they couldn’t know—and why then, if they couldn’t know, they were so confident it would be beneficial.

At one point, I tried a different approach, asking them instead to give examples of the downsides of the technology. This was a pillar of OpenAI’s founding mythology: The lab had to build good AGI before someone else built a bad one.

That tradition of anthropomorphizing continues to this day, aided by Hollywood tales combining the idea of “AI” with age-old depictions of human-made creations suddenly waking up. AI developers speak often about how their software “learns,” “reads,” or “creates” just like humans. Not only has this fed into a sense that current AI technologies are far more capable than they are, it has become a rhetorical tool for companies to avoid legal responsibility. Several artists and writers have sued AI developers for violating copyright laws by using their creative work—without their consent and without compensating them—to train AI systems. Developers have argued that doing so falls under fair use because it is no different from a human being “inspired” by others’ work. The omnipresent AI-to-human analogies have also fueled the sense that such software could become so capable that it surpasses us and comes to threaten our very existence. The fear of superintelligence is predicated on the idea that AI could somehow rise above us in the special quality that has made humans the planet’s superior species for tens of thousands of years.

This is where Te Hiku’s story diverges completely from OpenAI’s and Silicon Valley’s model of AI development. Intimately familiar with the devastating effects of colonial dispossession, Jones and Mahelona were determined to carry out the project only if they could guarantee three things—consent, reciprocity, and the Māori people’s sovereignty—at every stage of development. This meant that even before embarking on the project, they would get permission from the Māori community and their elders, asking them if the endeavor was even something they wanted; to collect the training data, they would seek contributions only from people who fully understood what the data would be used for and were willing to participate; to maximize the model’s benefit, they would listen to the community for what kinds of language-learning resources would be most helpful; and once they had the resources, they would also buy their own on-site Nvidia GPUs and servers to train their models without a dependency on any tech giant’s cloud.

In her 2019 talk at NeurIPS, during the Queer in AI workshop, Ria Kalluri, an AI researcher at Stanford, proposed an incisive alternative to the question of how to ensure AI does “good.” Goodness, benefit to humanity—these terms will always be in the eye of the beholder. Rather, we should ask how AI shifts power: Does it consolidate or redistribute that power? To put it in the frame of this book, does it continue to fortify the empire, or does it begin to wrest us back toward democracy?

In her talk, Kalluri raised the idea of different axes of power. This book touches on three: knowledge, resources, and influence. As it stands now, OpenAI and its competitor empires have control of each of them: through centralizing talent, eroding open science, and sealing their models from public scrutiny, they control knowledge production; through hoarding funding, data, labor, compute, energy, and land, they control and diminish other people’s resources; through creating and reinforcing ideologies and producing wildly popular demonstrations that captivate global imagination, they command far-reaching influence. Each of these reinforces the other. Controlling knowledge production fuels influence; growing influence accumulates resources; amassing resources secures knowledge production.