• Zihan Ding* - Princeton University (zhding[at]
  • Hao Dong - Peking University


In this chapter, we introduce a project named Arena for multi-agent reinforcement learning research. The hands-on instructions are provided in this chapter for building games with Arena toolkit, including a single agent game and a simple two-agent game with different reward schemes. The reward scheme in Arena is a way to specify the social structure among multiple agents, which contains social relationships of non-learnable, isolated, competitive, collaborative, and mixed types. Different reward schemes can be applied at the same time in a hierarchical structure in one game scene, together with the individual-to-group structure for physical units, to describe the complex relationships in multi-agent systems comprehensively. Moreover, we also show the process of applying the baseline in Arena, which provides several implemented multi-agent reinforcement learning algorithms as a benchmark. Through this project, we want to provide the readers with a useful tool for investigating multi-agent intelligence with customized game environments and multi-agent reinforcement learning algorithms.

Keywords: multi-agent reinforcement learning, learning environment, toolkit, competitive, collaborative, social relationship


Arena building toolkit is open-sourced here.

Arena baselines is open-sourced here.


To cite this book, please use this bibtex entry:

 title={Arena Platform for Multi-Agent Reinforcement Learning},
 author={Zihan Ding, Hao Dong},
 editor={Hao Dong, Zihan Ding, Shanghang Zhang},
 booktitle={Deep Reinforcement Learning: Fundamentals, Research, and Applications},
 publisher={Springer Nature},

If you find any typos or have suggestions for improving the book, do not hesitate to contact with the corresponding author (name with *).