SelfBlending: Artificial Intelligence-driven Augmentation with Hand Interactions for Seamless Reality Blending in Virtual Environments
- Type
- Article
- Citation
- IEEE Transactions on Visualization and Computer Graphics, v.32, no.7, pp.6958 - 6971
- Issued Date
- 2026-07
- Abstract
- Accessing real-world objects during immersive virtual reality (VR) experiences remains challenging, as current cross-reality systems often rely on predefined interaction steps, tracking devices/markers, or fixed object setups. They also lack support for personalized object recall, where users can add, remove, or modify real-world items blended into the virtual environment (VE). Many head-mounted displays (HMDs) include passthrough technology to switch between virtual and real worlds, but it often disrupts immersion by requiring a full shift from virtual to real. Thus, maintaining an optimal balance between virtuality and reality is difficult. To address these challenges, we developed SelfBlending, a framework that uses AI-based hand tracking to let users label physical objects through freehand gestures, then blends the selected item into the VE using object recognition, enabling interaction with the relevant real-world object. SelfBlending was evaluated against two common interaction conditions: the default passthrough feature in VR HMDs and the conventional approach of physically removing the HMD to access real-world objects. Results from seated, single-object interactions with tabletop-placed items showed that SelfBlending enhanced user experience by boosting presence, supporting efficient physical interaction, and improving cross-reality continuity. It also enabled selective interaction with real objects while minimizing the disruption of VR experience. © 1995-2012 IEEE.
- Publisher
- IEEE Computer Society
- ISSN
- 1077-2626
- Appears in Collections:
- Department of AI > 1. Journal Articles
- 파일 목록
-
-
관련 파일이 존재하지 않습니다.
-
Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.