Enhancing Esports Broadcasting with AI

Abstract

Esports has emerged as a major global industry, attracting millions of spectators who watch competitive video game tournaments. A crucial aspect of Esports broadcasts is the observer, responsible for controlling the in-game camera, selecting key moments, and presenting an engaging narrative to the audience. However, human observers face challenges such as the complexity of real-time decision-making, multitasking in fast-paced environments, and fatigue during extended broadcasts, which can result in critical moments being missed. To address these limitations, this thesis introduces an AI-based automatic observer system using deep learning techniques. The proposed system is developed and evaluated using StarCraft, a real-time strategy game known for its intricate gameplay and strategic depth. Leveraging deep learning’s ability to handle complex patterns, our approach aims to provide a more consistent and comprehensive spectating experience. The contributions of this research are threefold: First, we establish a systematic framework for data collection and preprocessing in StarCraft using game APIs. Second, we present a novel method that combines human observational data with the Mask R-CNN object detection model, allowing the system to identify and prioritize areas of interest similarly to human observers. Lastly, we introduce a hierarchical structure that integrates Vision Transformer-based (ViT) scene detection with a tracking mechanism, enhancing camera control stability and speed. This automatic observer system demonstrates the potential to revolutionize Esports broadcasts by delivering smooth, accurate, and engaging coverage of matches, even in high-intensity situations. The results highlight the effectiveness of AI-driven solutions in meeting the growing demands of the Esports industry.