Over the following year, Donald Trump’s spectacular rise and win in the 2016 US presidential election would shock the left-leaning workforce of the tech industry into self-reflection. As upheaval ripped through companies like Meta and Google and techlash sentiment gripped the public, AI researchers, too, began to question whether the field had moved too quickly to yoke its technologies to corporate bottom lines. An accounting of the societal impacts of commercializing AI research returned an unsettling scorecard: Automated software being sold to the police, mortgage brokers, and credit lenders were entrenching racial, gender, and class discrimination. Algorithms running Facebook’s News Feed and YouTube’s recommendation systems had likely polarized the public, fueled misinformation and extremism, enabled election interference, and, most horrifying in the case of Facebook, precipitated ethnic cleansing in Myanmar.

But this existential brand of AI safety, built on philosophical thought experiments, would soon come under fire as the AI research community awakened to the less apocalyptic and immediate real-world harms of AI. Around the same time Amodei published his paper, ProPublica published a groundbreaking investigation called “Machine Bias” that revealed algorithms were being used across the US criminal justice system in misguided attempts to predict future criminals, and those algorithms were classifying Black people as higher risk than white ones who had more extensive criminal records. The piece, and an overall souring on Big Tech post-2016 over the harms of social media, sparked a new wave of research reckoning with the harmful societal impacts of AI.

In a 2023 paper, Abeba Birhane and her coauthors would introduce the concept of “hate scaling laws” to critique the premise of training deep learning models on unfiltered data, or what they called “data-swamps.” They analyzed two publicly available image-and-text datasets used to train open-source image generators, LAION-400M and LAION-2B-en, both pulled from Common Crawl, with four hundred million and two billion images, respectively. They showed that the amount of hateful and abusive content scaled with the size of the dataset and exacerbated the discriminatory behaviors of the models trained on them. Models trained on the two billion images, for example, were five times more likely than models trained on the four hundred million images to label Black male faces as criminals. Later that year, a Stanford study analyzing LAION-5B, a dataset with five billion images used to train Stable Diffusion, would discover it contained thousands of images of verified and suspected child sexual abuse.

In 2018, the book Algorithms of Oppression by Safiya Umoja Noble, a professor of information, gender, and African American studies at the University of California, Los Angeles, had extensively documented the replication of racist worldviews in Google’s search results, such as by showing far more sexually explicit and pornographic content for “Black girls” than “white girls” and tropes about Black women being angry. Google at the time had used an older generation of language models to curate those results, which in extreme cases, Noble argued, may have also provoked racial violence.

A few hours later, I published a story for MIT Technology Review with the first detailed account of the paper’s contents. The signatories on the open letter would quickly double, reaching nearly 7,000 people from academia, civil society, and industry, including almost 2,700 Google employees. On December 9, as protests continued, Google CEO Sundar Pichai issued an apology. “We need to accept responsibility for the fact that a prominent Black, female leader with immense talent left Google unhappily,” he wrote. “Dr. Gebru is an expert in an important area of AI Ethics that we must continue to make progress on—progress that depends on our ability to ask ourselves challenging questions.”