Efficient Multi-Agent Reinforcement Learning for Many Agents

Abstract

Recently, multi-agent research systems have been used in the field of reinforcement learning to manage with cooperative agents. Simultaneously managing a large number of agents is challenging, so various approaches are being considered. Specifically, the application of high-dimension data methodologies is a significant challenge in the research of manyagent problems, as complexity increases exponentially with the number of agents. Furthermore, policy convergence can be difficult as the contribution of each agent is unclear. In this work, we flexibly decomposed a multi-agent problem into sub-multi-agent tasks using a clustering method, and applied this technique to a hierarchical structure. After abstracting the movements of units through hierarchical approach, a group’s action space and micro-control tasks were mapped onto highand low-level actions, respectively. We demonstrated our method through combat scenarios in the StarCraft video game. Our method successfully decomposed a complex multi-agent problem into homogeneous sub-tasks, and showed the advantage of making the training process efficient and inexpensive.