NetLogo User Community Models
Agent-based N-person prisoner's dilemma
by Michal Czaplinski (Submitted: 06/08/2011)
This model is an agent-based simulation of prisoner's dilemma. It allows the agents to evolve and be subject to cultural selection. The idea behind agent-based models like this one is to study phenomena that emerge from multiple interactions of many agents playing specific strategic games - in this case prisoner's dilemma.
Each of the agents can play one of 3 strategies: cooperate, defect or tit-for-tat.
Each round every agent follows roughly those steps:
The main focus of this simulation is to see how different rates of reproduction, death and cultural evolution as well as different parameters associated with these forces can affect the frequency of each strategy in the general population.
Each agent has a number of variables associated with him:
WHO - a unique number which each agent has.
STRATEGY - determines the strategy that a given agent will play.
SCORE - the sum of all payoffs that an agent received on all PD interactions.
NUM-GAMES* - the number of interactions that an agent had.
PARTNER-HISTORY - a list of all agents other than oneself indexed by their unique WHO number. Each entry on this list has a "true" or "false" value. This value depends on whether a particular agent cooperated or defected on the last interaction.
SCORE-LIST - a list of all payoffs that an agent received in all interaction in chronological order. For example, if the list is [3 3 0], it means that the agent got the payoff of 3 on the 1st and 2nd interaction and a payoff of 0 on the 3rd interaction.
SETUP - This button sets up the agents according to the numbers in the sliders on the left: n-cooperate, n-defect, n-tit-for-tat. The agents will appear on the big field on the right, where they will move around and interact. The field, by default, is a torus, which means that its sides fold onto each other. For example, when an agent goes beyond the top edge it will pop out at the very bottom.
GO - It runs the simulation. By default it will run forever, to stop you need to press the button again.
GO ONCE - runs the simulation for only one round.
At each round an agent has a chance to reproduce with some chance of random mutation.
(random number between 0 and 10000) / REPRODUCTION-RATE) <
If an agent is successful at reproduction, its child will inherit its STRATEGY and PARTNER-LIST. However, the "child" will have a new WHO number, a new SCORE-LIST and its SCORE will be set to 0. If a mutation occurs, the "child" agent will have a different STRATEGY.
There are 4 parameters which pertain to reproduction:
REPRODUCTION-RATE - is not in fact a rate, but a number which determines the range of numbers to which the output of the score function is compared. The probability that each agent will reproduce is proportional to this value.
REPRODUCTIVE-ADVANTAGE - determines the how steep the score function is. The higher the value, the steeper the function. For example, if two agents have a difference in average score of 0.5, the difference in reproductive success between them will be the greater, the higher value of REPRODUCTIVE-ADVANTAGE.
AGE-AT-REPRODUCTION - determines how many interactions do an agent have to have before it is allowed to reproduce.
MUTATION-RATE - the mutations can be turned on/off. Given that an agent will reproduce, this parameter determines (in percent) how likely the agent is to "hatch" an agent with a different STRATEGY than its own.
To prevent the populaton from growing exponentially agents have to die at a certain rate. The probability of death is determined by a quadratic fuction of a number of interactions that an agent had. There are two parameters associated with dying:
SURVIVAL-RATE - determines the probability that each agent will survive/die on a given round. The probability of death is the same for all agents with equal NUM-GAMES.
HARSHNESS - determines how difficult it is for each agent to survive by varying the shape of the function of NUM-GAMES. For example, let's assume that there are two agents: agent1 and agent2. The higher the HARSHNESS, the bigger will be the absolute difference in the probability of dying between agent1 and agent2.
An agent can change its strategy if it is "dissatisfied" with the payoffs that the present strategy has obtained in the last 5 rounds. The probability that an agent will change a strategy is a function of the sum of payoffs in the last 5 rounds. The lower the sum of payoffs in the last 5 rounds, the higher the probability that the agent will change its strategy. The decision between the other two strategies is random.
CULTURE-RATE - it determines the probability that an agent will change its strategy.
FLEXIBILITY - determines the shape of the function, i.e. how low an agent's score would have to be before it would have a very high chance of changing the strategy. For example, assume there are two agents with different sums of payoffs from the last 5 rounds. The absolute difference in the probability of changing a strategy between them will be greater the higher the value of FLEXIBILITY.
WHAT IT SHOWS?
The present model tries to simulate conditions which favor different strategies in a PD-type interactions among many agents. Some important assumptions made (other than those those imposed by many clear limitations of software's ability to simulate the complexity of social interactions) are as follows:
- The PARTNER-LISTS which allow agents (specifically tit-for-tats, because other strategies do not makes use of them) to "remeber" what each other agent did on the last interaction are inherited from "parent" agent to "child" agent. It is somewhat akin to inherited knowledge, as if the parents "taught" the "child" agent how to behave with different agents on the basis of their own experience. It is quite a liberal assumption, although I think that more realistic than not inheriting any knowledge at all. Were all the "child" agents to start from scratch, there would be no chance for tit-for-tats to outcompete defectors in the given setting, because each new tit-for-tat would have to get a beating from almost every defector in the population before it will learn to play defect with them.
- There is a number of variables which are not inherited, although they could be. The SCORE for example is not inherited because if it were, it would tend to skew the population towards higher and higher rates of reproduction.
- The space in which agents interact is small and limited and the agents move around randomly. It is quite a limiting assumtion, since most organisms do not really move truly randomly all of the time. On the other hand, it would be very biased to make agents of particular strategies "attracted'" to each other, because it would be easy to predict clustering and much higher rates of cooperation.
- The agents cannot recognize each other and the best they can do is remember the last interaction (tit-for-tats). This makes defection much easier.
RESULTS, CONCLUSIONS AND NEAT EXAMPLES
In the present model, the set of conditions under which tit-for-tat can outcompete defectors is very small. The circumstances which make it possible involve very low reproduction rates and very low death rates. This allows tit-for tats to be around for long enough to have interacted with most if not all defectors and "learned" that they need to defect when playing a defector. Low reproduction rates ensure that there are not too many new defectors entering the population, because each new defector would always take advantage of each of tit-for-tats. Moreover, the AGE-AT-REPRODUCTION would have to be very high, to allow the tit-for-tats to learn to defect when playing each of defectors in the initial population. To try you can use the values:
Cultural evolution does not seem to have much of an important effect on the overall distribution of strategies over time. If we allow agents to change strategies, the effect is somewhat randomizing regardless of whether the agents are allowed to reproduce or not. Without reproduction, the number of agents playing each strategy seems to simply oscillate around a particular value with, on average, defectors being the most common and cooperators being the least common. If reproduction is allowed, under more stringent conditions, the effect is almost the same as when there is no reproduction. Under "mild" conditions, all of the strategies seem to grow exponentially in number (or at least until the their number is too great for my computer to handle further computations).
POSSIBLE FURTHER EXTENSIONS
There is a number of ways in which the model could be improved and extended allowing for further exploration of the conditions which allow cooperation to evolve. For example:
- Cultural evolution could be made less random by letting agents switch to a more successul strategy with higher probability.
The model is partly based on the PD N-Person Iterated model developed by Uri Wilensky, available at:
This simulation was made possible thanks to free NetLogo software:
(back to the NetLogo User Community Models)