But this existential brand of AI safety, built on philosophical thought experiments, would soon come under fire as the AI research community awakened to the less apocalyptic and immediate real-world harms of AI. Around the same time Amodei published his paper, ProPublica published a groundbreaking investigation called “Machine Bias” that revealed algorithms were being used across the US criminal justice system in misguided attempts to predict future criminals, and those algorithms were classifying Black people as higher risk than white ones who had more extensive criminal records. The piece, and an overall souring on Big Tech post-2016 over the harms of social media, sparked a new wave of research reckoning with the harmful societal impacts of AI.

In a 2023 paper, Abeba Birhane and her coauthors would introduce the concept of “hate scaling laws” to critique the premise of training deep learning models on unfiltered data, or what they called “data-swamps.” They analyzed two publicly available image-and-text datasets used to train open-source image generators, LAION-400M and LAION-2B-en, both pulled from Common Crawl, with four hundred million and two billion images, respectively. They showed that the amount of hateful and abusive content scaled with the size of the dataset and exacerbated the discriminatory behaviors of the models trained on them. Models trained on the two billion images, for example, were five times more likely than models trained on the four hundred million images to label Black male faces as criminals. Later that year, a Stanford study analyzing LAION-5B, a dataset with five billion images used to train Stable Diffusion, would discover it contained thousands of images of verified and suspected child sexual abuse.

Using an auditing methodology Buolamwini developed for testing the discriminatory impact of computer-vision systems, the paper found that facial analysis software failed disproportionately on people of color, especially darker-skinned women. Buolamwini would subsequently produce a follow-on paper with Deborah Raji that, along with “Gender Shades,” would inspire a proliferation of related research, including an extensive US government audit citing and expanding on their findings. Two years later, widespread civil rights advocacy, spearheaded by Buolamwini with her newly founded organization Algorithmic Justice League, would lead Amazon, Microsoft, and IBM to ban their sales of facial recognition software to the police, the same month as OpenAI’s GPT-3 API launch.

In total, it presented four key warnings: First, large language models were growing so vast that they were generating an enormous environmental footprint, as found in Strubell’s paper.

Second, the demand for data was growing so vast that companies were scraping whatever they could find on the internet, inadvertently capturing more toxic and abusive language as well as subtler racist and sexist references.

Third, because such vast datasets were difficult to audit and scrutinize, it was extremely challenging to verify what was actually in them, making it harder to eradicate toxicity or more broadly ensure that they reflected evolving social norms and values. Finally, the model outputs were getting so good that people could easily mistake its statistically calculated outputs as language with real meaning and intent. This would make people prone not only to believing the text to be factual information but also to consider the model a competent adviser, a trustworthy confidant, and perhaps even something sentient.

For more than a year, the protests continued, picking up a second wave after Google fired Meg Mitchell less than three months later. Google said she had violated multiple codes of conduct; Mitchell had been downloading her emails and files related to Gebru’s ouster. Several Google employees, including Bengio, resigned; at least one conference and several researchers rejected Google’s sponsorship money. The company sought to stem the unending tide of criticism with the formation of a new center of expertise on responsible AI and public commitments to diversity. “This was a painful moment for the company,” a Google spokesperson said. “It reinforced how important it was that Google continue its work on responsible AI and learn from the experience.”