A Q-Learning Agent Discovers the Optimal Strategy Against Each Opponent in the Prisoner's Dilemma

This research explores whether a Q-learning agent can discover the right strategy to play against different types of opponents in the iterated Prisoner’s Dilemma, using only the points it earns each round. The agent is trained separately against four fixed strategies: Always Cooperate, Always Defect, Tit-for-Tat, and Grudger. The results show that the agent does not simply learn to cooperate, instead, it learns the optimal response depending on who it is playing against. It converges to near-complete cooperation only against Tit-for-Tat, and to defection against the other three. The most interesting case is Grudger, where the agent consistently fails to cooperate not because of a flaw in learning, but because its state representation cannot track enough history to recognize what Grudger is doing. This shows that what the agent learns is directly shaped by what it is able to observe.

Published on Wolfram Community. Research and essay © the author.