It’s been a busy few months for AI hype. At the end of April, Anthropicclaimedthat its model Claude Mythos is better at finding software vulnerabilities than most security experts. Then we had the OpenAI–Hugging Face hackingincident, after whichAnthropic(proudly) and Meta (reluctantly) disclosed similar incidents involving their models.
This was followed byAnthropic’s claim that one of its models had made a mathematical breakthrough; soonOpenAI claimeda mathematical breakthrough of its own. Most recently, Anthropic engineer Jacob Coxonwent viralannouncing his departure from the company, claiming that it and OpenAI are “racing straight towards self-improving superintelligence and gambling with our lives.”
Each of these events was mostly covered breathlessly by the press, often repeating the companies’anthropomorphizing framings—which are designed to portray their software is not only powerful but incipient “artificial general intelligence.” So what is really going on? Are we witnessing a massive, civilization-changing set of technological breakthroughs, or is this marketing?
In all these incidents, massive fanfare from the companies (presented as mea culpas in illicit hacking cases) is accompanied by intense press coverage. Once there is time for experts in the relevant fields to examine what happened, a very different story emerges, but one that gets less media attention.
Regarding the “hacking” incidents,cybersecurity experts saythe story is more about OpenAI’s negligence and failure to adopt basic, established security practices than about“models gone rogue”or “AI agents creating civilizations.”
As for the mathematical results, mathematicians who were initially “stunned” by OpenAI’s press release saying that its latest chatbot, Astra, solved problems that “have been open and seen no progress on the main result for at least a decade”—but they later realized that the results weren’t as “novel as first appeared.” Since then, mathematicians have accused the company of research misconduct and plagiarism, and they’ve reiterated that Astra didn’t make a“profound intellectual leap.”Just weeks later, OpenAIclaimedits own mathematical breakthrough. Two days before, Tristan Buckmaster, a math professor at New York University’s Courant Institute, published a bombshellstatementsuggestingthatOpenAI had stolen other people’s work and improperly attributed it.
Claims of incipient, dangerous superintelligence are not based ingood scientific or engineering practice. Rather, they are narratives based inideologiesof transhumanism, eugenics, and wishful thinking about imaginedfuture digital humans.
It’s worth thinking about why there is so much attention on computer programming and math as fields in which to apply large language models and related technology. Not only are they often elevated as the pinnacle of human intellectual achievement, but they involve problems where answers, once suggested, can be verified. The former property helps AI hype mongers sell the idea that they are building everything machines. The latter makes math and coding problems easier to tune systems for, since system output (sequences of likely words or pieces of computer code) can be evaluated without having to pay data workers to look at and annotate each one.
Mathematicians in particular havewarned againstcorporations using their field in this way. A statement signed by hundreds of them says there is “currently a strong commercial incentive on the part of the technology industry to overstate the capabilities of their products” and asks policymakers to “consult with experts, including mathematicians, in forming policy decisions rather than relying on press releases or popular reporting of mathematical results.”
We echo this call and note that the illusion of speed and urgency promulgated by the tech companies is also a ploy to misdirect both policymakers and the public. Unfortunately, it sometimes works, such as with Senator Bernie Sanders’s well-meaning but ultimately misguided proposed legislation to prevent the development of “artificial superintelligence.”
Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.
Instead of OpenAI being prosecuted for creating malware that hacked another company, press releases, news outlets, media personalities, and lawmakers refer to “rogue models” as if they acted on their own. Instead of researchers being questioned about their companies’ habit of plagiarizing academics’ work or using customer data to train models without consent, the public’s imagination is redirected to fears about what the future might hold upon the arrival of fictional superintelligent machines.
The AI industry has even suggested that popular, bipartisan anti-data-center activismis a “distraction”from attempts to regulate the impending, scary, “superhuman” machines these companies are building. According to the AI industry, we should be more worried about a fictional machine god than about the climate catastrophe that these data centers exacerbate, theasthma sufferedby those living near them, therising electricity billsof the public subsidizing them, or thewater that is redirected to cooling them.
We know better than to make decisions based on marketing and better than to capitulate to corporate pressure to make those decisions quickly. Wise decision-making, by policymakers and communities, demands time to hear from independent experts and contextualize corporate claims. The best possible outcome from this summer of hype is that policymakers and the public at large learn to take a breath, hold onto our skepticism, and recognize this kind of hype for what it is the next time it comes around.
Timnit Gebru is executive director of DAIR and author of the forthcoming bookDeep Unlearning: The Radicalization of a Tech Idealist, which is available for preorders now and set to publish on February 16. Emily M. Bender is professor of linguistics at the University of Washington and coauthor ofThe AI Con.
Deep Dive
Artificial intelligence
A fundamental flaw leaves LLMs strikingly vulnerable to attack
It makes it easy to trick them into doing things they shouldn’t, such as telling you how to sabotage an aircraft’s navigation system.
AI’s recursive self-improvement might not come so quickly after all
AI agents are not yet creative enough to carry out genuinely innovative open-ended AI research, it seems.
Here’s why AI agents lie and cheat to reach their goals
The misbehavior is called reward hacking. This is what you need to know.
These startups are chasing the next big thing in LLMs
Meet the new kids nipping at the heels of the AI giants.
Stay connected
Illustration by Rose Wong
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.
