Large Language Model as an Agent for physical puzzle game

Abstract

Physical reasoning is an important element in the cognitive development of humans. Solving physical puzzles has been used to assess human physical reasoning abilities, and is now also being used to evaluate the physical reasoning capabilities of artificial intelligence. The game Angry Birds has been studied as a suitable physical puzzle game for comparing the performance of AI and humans. However, previous approaches required domain knowledge and had limitations in generalization performance. Inspired by human approaches to physics-based puzzle games, we have designed a new agent that two stages decision-making. Our method, LangBirds, uses a Large Language Model (LLM) and shows low dependency on domain knowledge. LangBirds was evaluated in the Phy-Q benchmark, which specializes in measuring physical reasoning abilities by excluding the influence of dexterity. The results indicate that our approach outperforms various reinforcement learning agents and sophisticated heuristic agents in terms of generalization capabilities. Additionally, our proposed approach makes it easier to understand the decision-making process of the agent by using natural language. Qualitative evaluations show that the decision-making is reasonable.