The ‘WarGames’ Problem: Computer Science Has Long Understood What It Takes to Keep AI Under Control
The AI hacking events involving OpenAI, Anthropic, and Google underscore lessons that draw on years of computer science research.
Deven Desai
Oct 02, 2026
Image Credit
Egor Komarov on Unsplash
Share
AI agents don’t go rogue. That’s something only humans do.
Nevertheless, aNew York Timesarticle—representative of much news coverage of—described an OpenAI hackingas “AI bots going rogue and independently spearheading a cyberattack.”
Name-brand artificial intelligence agents have been on a hacking spree in 2026. OpenAI’s software agentshacked software company Hugging Faceandgovernment sites, Anthropic’sClaude hacked four companies’ systems, and in cybersecurity experiments Google’s Geminihacked three companies.
The AI companies areinvestigating tens of thousands of incidentsinvolving their agents, according to a report in Axios. These episodes have heightened fears about AI agents taking actions without human prompting.
The problem with headlines proclaiming that AI agents have gone rogue goes beyond anthropomorphizing the technology. It creates the impression that the agents were beyond the control of the AI companies that made them and there was little the companies could do about it.
As atechnology law and ethics scholarwho studies the effects disruptive technologies have on society, I know that’s not the case. If you don’t specifythe limits of what software is allowed to do, you should not be surprised when the software pursues all possible options to achieve its goal. This behavior—anAI pursuing a fixed objective—is what I call the “WarGames” problem, and it’s been recognized in the field of computer science for decades.
Been There, Seen That
In the 1983 movieWarGames, a teenager, David, hacks into a computer to play a new video game, Global Thermonuclear War. David doesn’t know that the computer is the government’s AI machine tasked with defending the United States from Russian nuclear attacks and can launch the US’s missiles. When David and his friend start the game,they select Las Vegas as the first target. While the North American Aerospace Defense Command goes on alert, launching bombers and warming up intercontinental ballistic missiles, David’s parents make him turn off the game. It’s over. Or is it?
The next day, David’sphone rings and he connects it to his computer. The caller is the government computer, which updates him that the game was interrupted, the primary goal has not yet been achieved, but a solution is expected in the next 52 hours. Like a modern software agent, the program has been running since David started the game and will work until the task is done.
Chess provides another view of the problem. Conquering chess was agoal for early AI. The rules of chess are well defined, including what winning looks like. So, programming a machine to play chess is straightforward. But imagine you let the software reason and act beyond the confines of the chessboard. The software might pursue options such as blackmailing its opponent or grabbing more compute time.
This example comes fromone of the most assigned textbookson AI, “Artificial Intelligence: A Modern Approach.” As the authors explain, you might be tempted to see those actions as rogue, but they “are a logical consequence of defining winning as the sole objective for the machine.”
What to Do About It
The AI hacking events involving OpenAI, Anthropic, and Google underscore a few lessons that draw on years of computer science research.
First, given the increasing use of AI agents, every organization involved in internet infrastructure, from large technology companies to small websites, needs to conduct audits and tighten up its internal security systems. As my colleagueMark Riedland I explain in our work onAIagents,application programming interfaces, or APIs, are a vital part of managing AI agents. APIs facilitate communication between different software systems. But as more people use AI agents, the agentsare likely to reveal and exploitpoor API construction and security.
Second, it’s important for AI agents to be designed to identify and authenticate themselves to third parties. What if yougave your AI agent your credentials? Website operators will need to know whether a human or bot ismakingareservation,selling a product, ormaking a purchase. They may want to limit automated systems that overwhelm their sites or reject AI agents because ofhigh rates of buying errors and refunds. Just as in laws covering human interactions, it’s important for third parties to be able to assess whom or what they are dealing with so they can allow or deny access.
Be Part of the Future
Sign up to receive top stories about groundbreaking technologies and visionary thinkers from SingularityHub.
100% Free.
No Spam.
Unsubscribe any time.
Third, it’s important for AI agents to have a default setting toslow down and check in with the human user. In the corporate AI hacking cases, the user appears to have launched their AI agents with the mistaken idea that the agents had a perfect specification of what to do and not to do. I believe it would have been better had it explored options and reported back to the user.
Google’s Gemini appears to have had a safeguard thatdetected the systemwas outside the simulated environment and so stopped its attacks. Slowing down and verifying actions, especially when a system detects it is exploiting a security hole, would be a big step in managing AI agents.
Fourth, AI companies could have strong controlsakin to those biomedical researchersuse, including ways to check what is happening and how the experiment is working. AI executives have claimed that their software is as ormore dangerous than fissionandcould end humanity. At the same time, they have not built safeguards commensurate with that level of risk.
Reality Check
At one point in “WarGames,” David asks the computer, called Joshua, whether it isstill playing the game. Joshua responds, “Of course.” It proceeds to update the time when it will launch its missiles and, much like a chatbot, asks, “Would you like to see some projected kill ratios?” David asks, “Is this a game? Or is it real?” Joshua replied, “What’s the difference?”
AI models, of course, don’t have any understanding of reality and are simply attempting to complete the tasks they’ve been assigned. Executives at AI companies, on the other hand, can’t claim that excuse.
As of September 2026, luck has so far prevailed. The AIs have attacked nonvital government sites and harmed smaller companies. If the AI companies—and government regulators—don’t take the “WarGames” problem seriously, I believe that we risk serious disasters. Tomorrow it could be taking out a hospital’s power system, wiping out a bank’s account system, breaking air traffic control, or worse.
Regarding theAI industry’s approachof rapidly developing powerful models, talking about the massive risks they pose, and at the same time failing to prevent harm, the movie’s climax offers a response: “A strange game. The only winning move is not to play.”
Disclosure statement: Deven Desai owns shares in Google, Inc. He has received unrestricted research gifts from Google, Inc. and Facebook, Inc. a decade ago. He has not been employed by Google since 2010.
This article is republished fromThe Conversationunder a Creative Commons license. Read theoriginal article.
Deven DesaiDeven DesaiDeven Desai is the Sue and John Staton Professor of Business Law and Ethics at the Georgia Institute of Technology, Scheller College of Business. He is also the associate director for law, policy, and ethics for ML@GATECH. He was the first and, to date, only academic research counsel at Google, and a visiting fellow at Princeton University’s Center for Information Technology Policy. As an undergraduate at Berkeley, he studied rhetoric and then went to Yale Law School for his JD. Deven’s work as a professor draws in part on his experience as a litigator handing intellectual property and technology matters with Quinn, Emanuel, Urquhart, & Sullivan, LLP; in-house counsel for an idealab! Internet infrastructure company; and as part of the policy and fundraising teams on the 2002 Cory Booker for Mayor campaign. Deven’s research focus on disruptive technologies’ effects on assumptions and equilibria in law and society. For example, he wrote some of the first work looking at how digital property affects privacy and inheritance law and the way 3D printing opens creativity while challenging core aspects of patent law. His most recent stream of research takes on critiques about fairness and bias in artificial intelligence and machine learning applying computer science theories such as information theory, network theory, and Rice’s Theorem to fashion viable law and policy solutions. He has also co-edited, with Mark Lemley, a special issue on scarcity, regulation, and the abundance society.
Related Articles
How Would AI Actually Kill All Humans? Here Are the Top 5 Scenarios
Toby Walsh
Sep 25, 2026
A Digital Cell Predicts Which Drugs Will Be Most Effective in Deadly Breast Cancer
Shelly Fan
Sep 24, 2026
Virtual Biotech Company Puts 37,000 AI Agents to Work on Drug Discovery
Edd Gent
Sep 18, 2026
Future
How Would AI Actually Kill All Humans? Here Are the Top 5 Scenarios
Toby Walsh
Sep 25, 2026
Biotechnology
A Digital Cell Predicts Which Drugs Will Be Most Effective in Deadly Breast Cancer
Shelly Fan
Sep 24, 2026
Biotechnology
Virtual Biotech Company Puts 37,000 AI Agents to Work on Drug Discovery
Edd Gent
Sep 18, 2026
What we’re reading
