RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch
Yizhong Zhang, Zhiqi Li, Sicheng Xu, Chong Li, Jiaolong Yang, Xin, Tong, Baining Guo

TL;DR
RemoteTouch introduces a dual representation system combining image-based and 3D geometry models to enable realistic hand touch interactions in immersive 3D video communication, enhancing user experience.
Contribution
The paper presents a novel dual representation approach for realistic hand rendering, allowing seamless hand contact in immersive 3D communication systems.
Findings
Enhanced realism in remote hand touch interactions
Improved immersive experience in 3D video communication
Consistent hand contact experience demonstrated through experiments
Abstract
Recent research advance has significantly improved the visual realism of immersive 3D video communication. In this work we present a method to further enhance this immersive experience by adding the hand touch capability ("remote hand clapping"). In our system, each meeting participant sits in front of a large screen with haptic feedback. The local participant can reach his hand out to the screen and perform hand clapping with the remote participant as if the two participants were only separated by a virtual glass. A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen. When the hand is very close to the screen, the RGBD data required for realistic rendering is no longer available. To tackle this challenge, we present a dual representation of the user's hand. Our dual representation not only…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVirtual Reality Applications and Impacts · Advanced Vision and Imaging · Tactile and Sensory Interactions
