In total, it presented four key warnings: First, large language models were growing so vast that they were generating an enormous environmental footprint, as found in Strubell’s paper.

Second, the demand for data was growing so vast that companies were scraping whatever they could find on the internet, inadvertently capturing more toxic and abusive language as well as subtler racist and sexist references.

Third, because such vast datasets were difficult to audit and scrutinize, it was extremely challenging to verify what was actually in them, making it harder to eradicate toxicity or more broadly ensure that they reflected evolving social norms and values. Finally, the model outputs were getting so good that people could easily mistake its statistically calculated outputs as language with real meaning and intent. This would make people prone not only to believing the text to be factual information but also to consider the model a competent adviser, a trustworthy confidant, and perhaps even something sentient.

The bottleneck for producing novel biological weapons, for example, is not about finding a recipe, Weinstein noted. Such recipes already abound online and are easily found via Google. It is about obtaining the materials and equipment to actually make the armaments. Restricting access to so-called frontier models would thus do little to fix this. But the collateral damage of suppressing the publication of AI models would risk weakening the foundations of US AI innovation. Open source—sharing and building on code and software released to the broader community—has long been the bedrock upon which the wealth of US-based startups flourish. Restricting model weights from being published would give smaller developers fewer pathways than ever to create their own AI products and services. It would further entrench the dominance of the giants represented in the Frontier Model Forum. AI models would also become ever harder to scrutinize, such as in the work of Sasha Luccioni, Yacine Jernite, and Emma Strubell, who have relied heavily on open generative AI models to quantify the carbon and environmental costs of continuing to scale them.

The prior month, in September 2023, Raji had found herself the singular academic, with financial ties neither to the industry nor Doomer community, testifying to Congress next to Altman, Musk, Nadella, Gates, Zuckerberg, Pichai, and Jack Clark, among other tech executives. They were all present for the very first of Senator Chuck Schumer’s AI Insight Forums, among the hottest and most consequential series of policy convenings that year to set in motion AI legislation. As her fellow witnesses spouted spectacular, unbacked claimsn about the promises and perils of AI, peppered with well-timed references to beating China that straightened the backs of attending senators, what shocked Raji the most was how much many in the audience appeared to buy into everything.

It dawned on her that the people sitting next to her, and their massive policy teams, had monopolized the message in Washington for so long that many policymakers now viewed it as gospel. A Schumer spokesperson would later note in the press that the senator was personally consulting with Altman and other OpenAI executives as he moved closer to regulation. “That for me was a huge realization,” Raji says. “Wow, we need more people just debunking—just looking at what people are saying and being like, ‘Actually, reality is more complicated.’”

The formula for dissolving empire thus requires the redistribution of power along each axis. The suggestions and recommendations I lay out here are exemplary but by no means comprehensive. First, to redistribute knowledge, we need greater funding to support its production outside the empire. That involves supporting researchers who can conduct independent evaluations of corporate models so we are not solely reliant on companies to understand their capabilities. It involves supporting organizations like DAIR that can pursue completely new directions of research, such as new forms of AI beyond large language models that are more efficient with data and energy. It involves supporting organizations like Te Hiku that can pursue task-specific, community-driven AI applications that strengthen marginalized communities.

With this transparency, we would also begin to redistribute power along our second axis: resources. By hiding the ingredients of their models as their intellectual property, the empires of AI have thus far been able to get away with seizing other people’s IP without credit, consent, or compensation. Visibility into company training data would make such extractive and exploitative behavior far more difficult. So, too, would visibility into company supply chains, including where they contract their labor and where they’re negotiating new leases of land to build more power plants and data centers, which so often happens under shell entities.