Predictively encoded graph convolutional network for noise-robust skeleton-based action recognition

Abstract

In skeleton-based action recognition, graph convolutional networks (GCNs), which model human body skeletons using graphical components such as nodes and connections, have recently achieved remarkable performance. While the current state-of-the-art methods for skeleton-based action recognition usually assume that completely observed skeletons will be provided, it is problematic to realize this assumption in real-world scenarios since the captured skeletons may be incomplete or noisy. In this work, we propose a skeleton-based action recognition method that is robust to noise interference for the given skeleton features. The key insight of our approach is to train a model by maximizing the mutual information between normal and noisy skeletons using predictive coding in the latent space. We conducted comprehensive skeleton-based action recognition experiments with defective skeletons using the NTU-RGB+D and Kinetics-Skeleton datasets. The experimental results demonstrate that when the skeleton samples are noisy, our approach achieves outstanding performances compared with the existing state-of-the-art methods.