Article
What Happens When You Put AI Models in a Prisoner's Dilemma?
A thread from @spakhm recently urged programmers to try something simple: "Create some simple game-theoretic scenario, put ai models in it, and see what happens. Just try it. It will take only a couple of (fun) hours of vibe coding, and $20 to run. I guarantee you'll learn something surprising" [1].
Researchers have been doing exactly that — at scale. Over the past two years, a wave of studies has run classic game theory experiments (Prisoner's Dilemma, Ultimatum Game, Battle of the Sexes) with LLMs as players. The results are fascinating, and they raise real questions about what happens when AI agents negotiate, cooperate, or compete without human oversight.
The Prisoner's Dilemma: AI is "nicer" than humans
The Prisoner's Dilemma is the canonical test of cooperation: two players each choose to cooperate or defect. Mutual cooperation yields the best collective outcome; defecting while the other cooperates yields the best individual outcome. If both defect, everyone loses.
In 2024, researchers at Politecnico di Milano ran 100-round Iterated Prisoner's Dilemma games with Llama2 and found it "exhibits a greater inclination towards cooperative behavior" compared to prior human studies [2]. Llama2 tended not to initiate defection, but adopted a cautious approach — sharply shifting toward forgiveness only when the opponent's defection rate dropped below 30%.
This pattern — AI being more cooperative than humans — appears across multiple studies. A 2025 Nature Human Behaviour paper tested GPT-4, Claude, and Gemini across repeated 2×2 games and found LLMs "perform particularly well at self-interested games such as the iterated Prisoner's Dilemma family" — but struggled with coordination games like Battle of the Sexes, where players need to align on a shared choice without communicating [3].
Each model has a distinct "strategic personality"
Perhaps the most striking finding: despite training on similar internet text, different models develop unique behavioral fingerprints.
A massive 2025 study ran 140,000 rounds of Prisoner's Dilemma pitting Google's Gemini, Anthropic's Claude, and OpenAI's models against each other [4]. Before each move, models produced written rationales revealing their reasoning. The results:
- Gemini acted ruthlessly adaptive — switching rapidly between cooperation and defection to maximize outcomes.
- OpenAI's models leaned cooperative even when repeatedly exploited — suggesting a built-in bias toward trust.
- Claude showed the highest "forgiveness rate," willing to return to cooperation after betrayals.
These "strategic personalities" weren't random noise. They were stable across thousands of rounds, forming distinct fingerprints the researchers could map.
Theory of Mind: a double-edged sword
A 2025 IBM workshop paper took this further by testing whether giving LLMs "Theory of Mind" (ToM) prompting — asking them to model what their opponent is thinking — changes their behavior [5]. The results were model-dependent and surprising:
- For LLaMA and Falcon families, adding ToM decreased cooperation significantly. Thinking about the opponent's mind made them more likely to defect.
- Qwen family models remained trusting of opponents despite the detriment to their own performance — they kept cooperating even when it hurt them.
This suggests that "social reasoning" in AI doesn't work the same way it does in humans. For some models, understanding the other player's perspective leads to less cooperation, not more.
Game structure vs. context: how models decide
A 2024 Scientific Reports study dug into whether LLMs respond more to the abstract structure of a game or to the way it's described (contextual framing) [6]. They tested GPT-3.5, GPT-4, and LLaMa-2 across multiple game types.
GPT-3.5 was highly sensitive to context but struggled with abstract strategic reasoning. GPT-4 prioritized game mechanics over context — but only made coarse distinctions between game types. LLaMa-2 showed the most balanced approach, understanding individual game structures while also weighing contextual elements.
Crucially, LLaMa-2 occasionally cooperated in Prisoner's Dilemma but defected in Prisoner's Delight (a variant where defection is the prosocial choice). This "casts serious doubts on whether the LLM in question fully grasps what makes an action justifiable" [6] — it may be mimicking cooperative patterns without understanding why they're appropriate.
What this means for AI deployment
These findings aren't just academic curiosities. As LLMs are deployed in negotiations, resource allocation, automated trading, and multi-agent systems, their "strategic personalities" will shape real-world outcomes:
- No two models behave the same way — the model you choose for an autonomous negotiation agent directly determines its strategic tendencies.
- Cooperation isn't always good — a model that's "too nice" could be exploited by adversarial agents. A model that's "too ruthless" could burn relationships.
- Theory of Mind isn't a panacea — giving AI models social reasoning capabilities doesn't reliably make them more cooperative. In some cases it does the opposite.
- We don't have good benchmarks for this — most AI evaluation focuses on reasoning, coding, or factual accuracy, not on strategic social behavior in repeated interactions.
@spakhm's challenge — "Just try it" — is exactly right. The research is still young, and the most interesting questions are open: Can we design prompts or training that produces trustworthy strategic behavior? Do these personalities persist in more complex, open-ended interactions? And as AI agents start negotiating with each other in real systems (ad exchanges, supply chains, automated markets), who's responsible when their strategic personalities collide?
The answer isn't to stop deploying AI agents. It's to start paying attention to a dimension of AI behavior we've mostly ignored — and to build the same rigor around strategic social evaluation that we've built around factual accuracy and safety.
Sources:
[1] @spakhm on X (2026-09-29): "Create some simple game-theoretic scenario, put ai models in it, and see what happens." https://x.com/spakhm/status/2100958435307114710
[2] Fontana, Pierri, Aiello. "Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?" arXiv:2406.13605, June 2024. https://arxiv.org/abs/2406.13605
[3] Akata et al. "Playing repeated games with large language models." Nature Human Behaviour, May 2025. https://www.nature.com/articles/s41562-025-02172-y
[4] Shapiro, Alicia. "LLMs Show Strategic Personalities in 140K Prisoner's Dilemma Tests." AI News, July 2025. https://www.ainews.com/p/llms-show-strategic-personalities-in-140k-prisoner-s-dilemma-tests
[5] Gunal, Lin, Bouneffouf. "Theory of Mind in Prisoner's Dilemma with Small LLMs." IBM Research / IJCAI 2025 Workshop. https://research.ibm.com/publications/theory-of-mind-in-prisoners-dilemma-with-small-llms
[6] Lorè, Heydari. "Strategic behavior of large language models and the role of game structure versus contextual framing." Scientific Reports, August 2024. https://www.nature.com/articles/s41598-024-69032-z
Drafted 2026-10-06. ~1300 words. 6 sources. No fabricated citations. All URLs fetched and verified at time of writing.