LLM-Assisted Reinforcement Learning for Strategic Decision-Making in the Persian Incursion Wargame

Abstract

Strategic wargames are challenging for artificial intelligence （AI） agents because they require legal decision making, long- term planning, uncertain outcomes, and several possible paths to victory. This thesis studies Persian Incursion as a rules-based executable wargame environment for evaluating whether an AI agent can make legal and strategically useful decisions when only limited gameplay data is available. Instead of treating the game as a free-form text interaction, the environment provides structured game states, legal action menus, legality checks, and state-transition records that show how each action changes the game. This design makes it possible to evaluate agents not only by whether they win, but also by whether their actions are legal, meaningful, and connected to a clear victory plan. The thesis proposes a hierarchical legal-action decision framework that breaks gameplay into smaller strategic decisions, including selecting a branch, choosing a victory route, identifying subgoals, ranking targets, selecting a legal macro action, and executing verifier-gated game actions. A central result is that this framework moves the agent beyond the zero-terminal-win regime observed in flatter legal-action approaches. Early agents could often remain legal and make locally plausible moves, but they rarely converted progress into completed victories. By changing the decision interface from direct action prediction to route-aware macro selection and legal follow-through, the agent can produce executable paths to terminal outcomes while preserving legality. The experiments evaluate supervised hierarchical policies, route-aware macro controllers, and macro- level arbitration layers across nuclear, oil, and political victory routes under stage-conditioned Red stress profiles. The results show large improvements over passive and weaker policy baselines in legal macro selection, route-aware behavior, victory-proximity progress, and terminal-win achievement. However, strict replay analysis also shows that aggregate win rate alone can be misleading. Many successful outcomes are driven by political terminal paths, while robust nuclear and oil terminal victories remain much harder to achieve. To make this distinction visible, the thesis introduces diagnostic metrics beyond simple win rate, including raw terminal game-over cause, route-specific terminal wins, non-political win rate, final distance to victory, progress over time, avoidable pass behavior, nuclear-site Sanghyun Choi. LLM-Assisted Reinforcement Learning for Strategic Decision-Making in the Persian Incursion Wargame（페르시안 인커젼 워게임의 전략적 의사결정을 위 한 대규모 언어모델 보조 강화학습 연구）, Department of Electrical Engineering and Computer Science, College of Information and Computing. 2026. 52p. Advisor: Prof. Hongkook Kim MS/EC 20241162 progress, required nuclear-roll distance, and modifier progression. The results support a bounded but important claim. The proposed framework does not solve Persian Incursion as a fully general adversarial wargame, and decisive campaign closure remains difficult. However, it demonstrates how a verifier-gated hierarchical agent can break out of a zero-win regime, maintain legal structured decision making, discover executable victory paths, and expose the difference between shortcut-driven terminal wins and durable campaign-level mastery. Overall, this thesis contributes a rules-based executable wargame environment, a hierarchical framework for legal strategic decision making, strict diagnostic metrics for long-horizon victory progress, and an evaluation methodology for separating genuine campaign success from artifact-driven or route- specific performance.