State Grouping based Genetic-MCTS Hybrid Approach for Deep Reinforcement Learning

Abstract

Although reinforcement learning has recently been recognized as a promising means of machine learning via sequences of trial and error without preexisting big data, its application remains limited for real-world problems due to the long initial learning time, high computational complexity, and learning instability. In order to alleviate such difficulty, researchers devised several measures to improve the learning stability for conditionally constrained problems, including Transition learning, Imitation learning, and Trust region policy optimization. Nonetheless, problems consisting of numerous branching elements with real-time properties still persist, requesting a new paradigm for reinforcement learning. In this paper, we propose a new SG-Hybrid AI in conjunction with Distributed deep reinforcement learning. Its core principle is to divide a huge state space into several independent state groups using space grouping, exploit higher priority learning behaviors with a Genetic-MCTS hybrid method, and build a good behavior set to effectively reduce the branching complexity. The experimental results on a real-time fighting game have proven the effectiveness of our proposed approach.