Embedded Physical AI for Virtual Locomotion: Gesture Classification via Tactile Sensor

Abstract

Virtual reality (VR) locomotion systems frequently struggle to balance spatial constraints, user immersion, and physical comfort, often relying on cumbersome wearable devices. This thesis proposes a non-intrusive, tactile sensor-based framework for VR navigation, centered on embedded physical AI. The core contribution is a robust foot-based gesture classification system utilizing a high-resolution carpet-type tactile sensor. To precisely interpret complex spatiotemporal pressure patterns, we introduce the Self-Teaching Vision Transformer (STVIT). By integrating hierarchical patch partitioning with a unique self-distillation mechanism, where the model continuously learns from its own prior iterations, STVIT achieves state-of-the-art accuracy in distinguishing nuanced, in-place actions like marching and sneaking. This approach demonstrably enhances user control and virtual presence. Expanding this tactile paradigm to seated environments, we present the Leg-swing interface. By embedding a high-resolution pressure sensor directly into a chair's seat, the system maps natural leg-swinging and torso leaning to virtual movement. This interface establishes an optimal middle ground, mitigating the severe physical fatigue of standard seated walking-in-place techniques while providing significantly higher immersion than traditional joystick controllers. Finally, to ensure system robustness across diverse user demographics, we introduce the Physique-Guided Swin Transformer (PG-Swin). This architecture dynamically recalibrates spatiotemporal features using low-dimensional physical metadata via a multiplicative gating mechanism. Collectively, these methodologies establish a highly immersive, comfortable, and robust locomotion framework for virtual environments.