If you find any typos or have suggestions for improving the book, do not hestitate to contact us via email at: hao.dong[at].pku.edu.cn
Content
The following chapters introduce selected research topics in deep reinforcement learning, which would be useful if the readers would like to have a deeper understanding or start the related research.
We first discuss several challenges of deep reinforcement learning in Chap. 7, including sample efficiency, learning stability, catastrophic interference, exploration, meta-learning and representation learning, multi-agent reinforcement learning, sim2real, and large-scale reinforcement learning. Then we compose six chapters to introduce different advanced research topics and algorithms in deep reinforcement learning, as well as indicating how they are related to and applied for solving those challenges. From a research perspective, lots of recent advances are introduced in this part of contents with seven chapters in total.
Chapter 8 introduces the imitation learning in relatively comprehensive perspectives. Combination of imitation learning and reinforcement learning helps to alleviate the challenge of low sample efficiency in deep reinforcement learning, through leveraging the expert demonstrations in the learning process.
Chapter 9 introduces model-based reinforcement learning, which also improves the learning efficiency in deep reinforcement learning, but through leveraging the models of environments. Model-based reinforcement learning is a worthwhile direction with fruitful advanced contents when facing real-world applications, as well as a frontier hotspot.
Chapter 10 describes hierarchical reinforcement learning, which helps with problems including catastrophic interference and hard exploration in deep reinforcement learning, as well as improving the overall learning efficiency. Options framework and feudal reinforcement learning are highlighted in the description.
Chapter 11 describes the concept of multi-agent reinforcement learning, as an extension of reinforcement learning to tasks with more than one agent. Competitive and collaborative relationships among agents, Nash equilibrium and some multi-agent reinforcement learning algorithms are detailed in this chapter.
Chapter 12 introduces parallel computing in deep reinforcement learning, for solving the scalability challenge and improving the learning speed in wall-clock time. Different parallel training frameworks are introduced in this chapter, which helps to employ deep reinforcement learning in large-scale real-world applications. The related codes are released in the following link: https://github.com/deep-reinforcement-learning-book.