Within two days, Bender had sent Gebru an outline. They later came up with a title, adding a cheeky emoji for emphasis: âOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big?â
Gebru assembled a research team for the paper within Google, including her colead Mitchell. In response to the encouraging words in Deanâs annual review, she flagged the paper as an example of the work she was pursuing. âDefinitely not my area of expertise,â Dean said, âbut would definitely learn from reading it.â
Related Quotes
On the Thursday a week before Thanksgiving, after Gebru had submitted the paper to the conference, she received a calendar invite without explanation to meet Megan Kacholia, Google Researchâs VP of engineering, over a video call less than three hours later. The meeting lasted only thirty minutes, and Kacholia cut to the point: Gebru needed to retract the paper.
The request was a dramatic aberration from the way Google and the rest of the industry handled research. Like many labs at other companies, Google Brain had until then largely conducted itself as an academic operation and given researchers wide latitude to pursue the questions they wanted to. At times, the company reviewed papers to ensure they didnât expose sensitive IP or customer data. But researchers like Gebru had never known the company to block or retract a paper simply for shedding light on inconvenient truths. That Google was even willing to pull this move, some researchers would later reflect, was not only because of the new competitive pressure from OpenAI but also because of the work OpenAI had done to legitimize withholding research after GPT-2.
A few hours later, I published a story for MIT Technology Review with the first detailed account of the paperâs contents. The signatories on the open letter would quickly double, reaching nearly 7,000 people from academia, civil society, and industry, including almost 2,700 Google employees. On December 9, as protests continued, Google CEO Sundar Pichai issued an apology. âWe need to accept responsibility for the fact that a prominent Black, female leader with immense talent left Google unhappily,â he wrote. âDr. Gebru is an expert in an important area of AI Ethics that we must continue to make progress onâprogress that depends on our ability to ask ourselves challenging questions.â
But what seemed to bother Dean the most was how other people had misread Strubellâs research to make Google look significantly worse. The âStochastic Parrotsâ paper, Dean argued, risked exacerbating this issue. Because Gebru did have access to Googleâs internal numbers and was citing Strubellâs external estimate anyway, it could appear as if Strubellâs calculations were an accurate reflection of the companyâs emissions. To Dean, this justified his and other senior executivesâ criticisms of Gebruâs paper: If Gebru had wanted to cite Strubell, she should have chosen an estimate that was not Googleâs Evolved Transformer; if Gebru had wanted to cite the Evolved Transformer, she should have sought internal Google numbers.
Some researchers found this logic frustratingly inadequate. Google had never made those internal numbers public previously, even in response to Strubellâs original paper; now it was blaming Gebru for its own lack of transparency while also refusing to let her cite publicly available estimates based on legitimate assumptions. Never mind that the company had unceremoniously forced out Gebru before sheâd even had a chance to consult internal numbers and revise her paper. The only possible outcome of this catch-22 was censorship of critical accountability research.
Nitasha Tiku, the reporter who broke the story, also spoke to Emily Bender and Meg Mitchell, who had warned in their âStochastic Parrotsâ paper about the problem of large language models fooling people into seeing real meaning and intent behind their generations. âWe now have machines that can mindlessly generate words, but we havenât learned how to stop imagining a mind behind them,â said Bender.
âIâm really concerned about what it means for people to increasingly be affected by the illusion,â when that illusion is becoming so good, Mitchell said.
Alex Hanna, a sociologist and one of the Google coauthors on the âStochastic Parrotsâ paper, became the first to join Gebru as DAIRâs director of research. Hannaâs first order of business was to write a research philosophy to further elaborate the ethos of the organizationâs work. To do so, Gebru and Hanna hired DAIRâs third person, Milagros Miceli, another sociologist and computer scientist who had been conducting research into the AI industryâs exploitative labor practices. Together they wrote their philosophy: âOur research is intended to benefit communities which are typically not served by AI and create pathways to refuse, interrogate, and reshape AI systems together.â