Observer: An open-source framework for automating spectator for Real-time Strategy game of StarCraft

Abstract

Selecting engaging scenes is a critical component of esports broadcasting, traditionally performed by human observers. While recent research has explored AI-based automation, existing approaches often lack comprehensive frameworks for data extraction, human behavior modeling, and interface integration. We present Observer , an open-source framework that collects and preprocesses raw in-game data from StarCraft along with human observer viewport data to train AI-based automatic observers. The system transforms gameplay into multi-channel (hereafter, feature channels) representations and uses a modified Intersection over Union (IoU) metric to evaluate the overlap between predicted and aggregated human viewports. As reported in prior work, a learned observer achieves 56.9% similarity to human behavior, surpassing representative rule-based methods (52.4% and 49.1%) on standard benchmarks. In this software paper, we focus on a standardized, reproducible pipeline and system-level metrics. © 2025 Elsevier B.V., All rights reserved.