NetLogo User Community Models

(back to the NetLogo User Community Models)

Download If clicking does not initiate a download, try right clicking or control clicking and choosing "Save" or "Download".(The run link is disabled for this model because it was made in a version prior to NetLogo 6.0, which NetLogo Web requires.)

INTRODUCTION

This model is an agent-based simulation of prisoner's dilemma. It allows the agents to evolve and be subject to cultural selection. The idea behind agent-based models like this one is to study phenomena that emerge from multiple interactions of many agents playing specific strategic games - in this case prisoner's dilemma.

Each of the agents can play one of 3 strategies: cooperate, defect or tit-for-tat.
The number of agents playing each strategy can be determined by N-COOPERATE, N-DEFECT and N-TIT-FOR-TAT sliders.
All agents currently playing a particular strategy have different colors:
cooperators - blue
defectors - red
tit-for-tats - green

Each round every agent follows roughly those steps:
1) move in a random direction
2) play prisoner's dilemma if there is another agent nearby
3) reproduce
4) change strategy due to "cultural evolution"
5) die

The main focus of this simulation is to see how different rates of reproduction, death and cultural evolution as well as different parameters associated with these forces can affect the frequency of each strategy in the general population.

AGENTS

Each agent has a number of variables associated with him:

WHO - a unique number which each agent has.

STRATEGY - determines the strategy that a given agent will play.

SCORE - the sum of all payoffs that an agent received on all PD interactions.

NUM-GAMES* - the number of interactions that an agent had.

PARTNER-HISTORY - a list of all agents other than oneself indexed by their unique WHO number. Each entry on this list has a "true" or "false" value. This value depends on whether a particular agent cooperated or defected on the last interaction.
This list allows tit-for-tats to "remember" what each other agent did on the last interaction and adjust their action accordingly. For example, if a particular tit-for-tat interacts with another agent with (WHO = 11) it will look up the 11th item on this list and depending on the value choose to cooperate or defect.

SCORE-LIST - a list of all payoffs that an agent received in all interaction in chronological order. For example, if the list is [3 3 0], it means that the agent got the payoff of 3 on the 1st and 2nd interaction and a payoff of 0 on the 3rd interaction.

BUTTONS

SETUP - This button sets up the agents according to the numbers in the sliders on the left: n-cooperate, n-defect, n-tit-for-tat. The agents will appear on the big field on the right, where they will move around and interact. The field, by default, is a torus, which means that its sides fold onto each other. For example, when an agent goes beyond the top edge it will pop out at the very bottom.

GO - It runs the simulation. By default it will run forever, to stop you need to press the button again.

GO ONCE - runs the simulation for only one round.

REPRODUCTION

At each round an agent has a chance to reproduce with some chance of random mutation.
The probablity that an agent will reproduce is proportional to the average score of the agent, which is calculated by dividing SCORE of each agent by NUM-GAMES.
The average score is then fed into a score function to produce a value which is subsequently compared to a number determined by REPRODUCTION-RATE.
The agent will reproduce if:

(random number between 0 and 10000) / REPRODUCTION-RATE) <
(average score) ^ 2) / 20) * REPRODUCTIVE-ADVANTAGE

If an agent is successful at reproduction, its child will inherit its STRATEGY and PARTNER-LIST. However, the "child" will have a new WHO number, a new SCORE-LIST and its SCORE will be set to 0. If a mutation occurs, the "child" agent will have a different STRATEGY.

There are 4 parameters which pertain to reproduction:

REPRODUCTION-RATE - is not in fact a rate, but a number which determines the range of numbers to which the output of the score function is compared. The probability that each agent will reproduce is proportional to this value.

REPRODUCTIVE-ADVANTAGE - determines the how steep the score function is. The higher the value, the steeper the function. For example, if two agents have a difference in average score of 0.5, the difference in reproductive success between them will be the greater, the higher value of REPRODUCTIVE-ADVANTAGE.

AGE-AT-REPRODUCTION - determines how many interactions do an agent have to have before it is allowed to reproduce.

MUTATION-RATE - the mutations can be turned on/off. Given that an agent will reproduce, this parameter determines (in percent) how likely the agent is to "hatch" an agent with a different STRATEGY than its own.

DYING

To prevent the populaton from growing exponentially agents have to die at a certain rate. The probability of death is determined by a quadratic fuction of a number of interactions that an agent had. There are two parameters associated with dying:

SURVIVAL-RATE - determines the probability that each agent will survive/die on a given round. The probability of death is the same for all agents with equal NUM-GAMES.

HARSHNESS - determines how difficult it is for each agent to survive by varying the shape of the function of NUM-GAMES. For example, let's assume that there are two agents: agent1 and agent2. The higher the HARSHNESS, the bigger will be the absolute difference in the probability of dying between agent1 and agent2.

CULTURAL EVOLUTION

An agent can change its strategy if it is "dissatisfied" with the payoffs that the present strategy has obtained in the last 5 rounds. The probability that an agent will change a strategy is a function of the sum of payoffs in the last 5 rounds. The lower the sum of payoffs in the last 5 rounds, the higher the probability that the agent will change its strategy. The decision between the other two strategies is random.
There are 2 parameters:

CULTURE-RATE - it determines the probability that an agent will change its strategy.

FLEXIBILITY - determines the shape of the function, i.e. how low an agent's score would have to be before it would have a very high chance of changing the strategy. For example, assume there are two agents with different sums of payoffs from the last 5 rounds. The absolute difference in the probability of changing a strategy between them will be greater the higher the value of FLEXIBILITY.

WHAT IT SHOWS?

The present model tries to simulate conditions which favor different strategies in a PD-type interactions among many agents. Some important assumptions made (other than those those imposed by many clear limitations of software's ability to simulate the complexity of social interactions) are as follows:

- The PARTNER-LISTS which allow agents (specifically tit-for-tats, because other strategies do not makes use of them) to "remeber" what each other agent did on the last interaction are inherited from "parent" agent to "child" agent. It is somewhat akin to inherited knowledge, as if the parents "taught" the "child" agent how to behave with different agents on the basis of their own experience. It is quite a liberal assumption, although I think that more realistic than not inheriting any knowledge at all. Were all the "child" agents to start from scratch, there would be no chance for tit-for-tats to outcompete defectors in the given setting, because each new tit-for-tat would have to get a beating from almost every defector in the population before it will learn to play defect with them.

- There is a number of variables which are not inherited, although they could be. The SCORE for example is not inherited because if it were, it would tend to skew the population towards higher and higher rates of reproduction.

- The space in which agents interact is small and limited and the agents move around randomly. It is quite a limiting assumtion, since most organisms do not really move truly randomly all of the time. On the other hand, it would be very biased to make agents of particular strategies "attracted'" to each other, because it would be easy to predict clustering and much higher rates of cooperation.

- The agents cannot recognize each other and the best they can do is remember the last interaction (tit-for-tats). This makes defection much easier.

RESULTS, CONCLUSIONS AND NEAT EXAMPLES
The main conclusion from the simulation is that over a very wide range of parameters, defectors outcompete both other strategies by a wide margin. It seems that the assumptions make it extremely hard for cooperation to emerge as a successful strategy. For example one can try setting all the parameters at intermediate values and AGE-AT-REPRODUCTION at about 10-20. The defectors will outcompete other strategies very quickly. In fact, if the parameters are set to make life in the simulation "easy" with high survival rates, low age of first reproduction and high reproduction rates, the number of defectors will grow exponentially and vastly surpass the number of other strategies in

In the present model, the set of conditions under which tit-for-tat can outcompete defectors is very small. The circumstances which make it possible involve very low reproduction rates and very low death rates. This allows tit-for tats to be around for long enough to have interacted with most if not all defectors and "learned" that they need to defect when playing a defector. Low reproduction rates ensure that there are not too many new defectors entering the population, because each new defector would always take advantage of each of tit-for-tats. Moreover, the AGE-AT-REPRODUCTION would have to be very high, to allow the tit-for-tats to learn to defect when playing each of defectors in the initial population. To try you can use the values:

20 N-TIT-FOR-TAt
20 N-DEFECTORS
3 REPRODUCTION-RATE
10 REPRODUCTION-ADVANTAGE
100 AGE-AT-REPRODUCTION
off MUTATIONS
100 SURVIVAL-RATE
1 HARSHNESS
off CULTURAL EVOLUTION

Cultural evolution does not seem to have much of an important effect on the overall distribution of strategies over time. If we allow agents to change strategies, the effect is somewhat randomizing regardless of whether the agents are allowed to reproduce or not. Without reproduction, the number of agents playing each strategy seems to simply oscillate around a particular value with, on average, defectors being the most common and cooperators being the least common. If reproduction is allowed, under more stringent conditions, the effect is almost the same as when there is no reproduction. Under "mild" conditions, all of the strategies seem to grow exponentially in number (or at least until the their number is too great for my computer to handle further computations).
Interestingly, even though cultural evolution does not seem to favor any strategy, it makes cooperation possible by preventing the cooperative strategies from being eliminated from the general population.

POSSIBLE FURTHER EXTENSIONS

There is a number of ways in which the model could be improved and extended allowing for further exploration of the conditions which allow cooperation to evolve. For example:

- Cultural evolution could be made less random by letting agents switch to a more successul strategy with higher probability.
- An area efect, where agents would be more likely to switch to a strategy that the agents nearby play would also be interesting to explore.
- The agents could be made to inherit the PARTHNER-LISTS with some degree of error, instead of as perfect copies of their parent's PARTNER-LISTS, which would be more realistic.
- Of course, other strategies such as Pavlov or generous tit-for-tat could be included in the model.
- It would be possible to make agents "recognize" each other before each interaction with some probability of an incorrect judgement. That could allow avoiding the sucker's payoff from defectors, but also making mistakes in interactions with other "nice" strategies.

*NOTE
NUM-GAMES is in fact NUM-<strategy>-GAMES in the code. For each strategy the variable has a different name, with the name of the strategy in place of <strategy>. I used NUM-GAMES for sake of simplicity.

CREDITS

The model is partly based on the PD N-Person Iterated model developed by Uri Wilensky, available at:
- Wilensky, U. (2002). NetLogo PD N-Person Iterated model. http://ccl.northwestern.edu/netlogo/models/PDN-PersonIterated. Center for Connected

This simulation was made possible thanks to free NetLogo software:
Learning and Computer-Based Modeling, Northwestern University, Evanston, IL.
- Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL.