Leveraging Self-Attention for Multimodal Skeleton-Based Video Anomaly Detection

Abstract

Anomaly detection is crucial in areas like surveillance, healthcare, and manufacturing. One-class classification (OCC) methods are commonly used for this purpose, as they are trained solely on normal data due to the rarity of anomalies. Traditional OCC methods establish a specific boundary for normal behaviors and identify anomalies as deviations from this boundary. However, they fail to account for the wide variation in normal behaviors, as people can act in numerous ways, complicating anomaly detection. In response to this challenge, our research presents an innovative method for detecting anomalies in video footage. Our method identifies rare events and considers the diverse nature of normal and abnormal human behaviors. By utilizing diffusion-based probabilistic models and integrating hierarchical attention mechanisms, the proposed technique generates potential future human poses. It leverages past motion history to predict a broad range of future movements, boosting the model's ability to identify anomalies. The proposed technique creates a variety of possible future motions, aiding in the understanding of potential future scenarios. Anomalies are detected when the generated motions significantly deviate from expected future behavior, enabling early intervention. We conducted extensive experiments on the test sets of the UBnormal, HR-STC, and HR-Avenue datasets to show the advantage of our technique over current methods. Our methodology not only improves frame-by-frame detection accuracy in complex dynamic scenarios but also holds promise for practical application in real-world situations.