VirtualCube: An Immersive 3D Video Communication System
Yizhong Zhang, Jiaolong Yang, Zhen Liu, Ruicheng Wang, Guojun Chen,, Xin Tong, and Baining Guo

TL;DR
VirtualCube is an immersive 3D video conferencing system that uses RGBD cameras and advanced rendering algorithms to enable realistic, eye-contact-preserving remote communication with shared workspaces.
Contribution
It introduces VirtualCube, a standardized, hardware-agnostic 3D conferencing platform with real-time rendering and gaze awareness capabilities.
Findings
Real-time rendering with multi-view stereo and Lumi-Net improves visual quality.
The system preserves mutual eye gaze, enabling natural eye contact.
Supports shared workspaces and attention tracking for remote collaboration.
Abstract
The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies. The key ingredient is VirtualCube, an abstract representation of a real-world cubicle instrumented with RGBD cameras for capturing the 3D geometry and texture of a user. We design VirtualCube so that the task of data capturing is standardized and significantly simplified, and everything can be built using off-the-shelf hardware. We use VirtualCubes as the basic building blocks of a virtual conferencing environment, and we provide each VirtualCube user with a surrounding display showing life-size videos of remote participants. To achieve real-time rendering of remote participants, we develop the V-Cube View algorithm, which uses multi-view stereo for more accurate depth estimation and Lumi-Net rendering for better rendering quality. The VirtualCube system…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Visual Attention and Saliency Detection · Advanced Image and Video Retrieval Techniques
MethodsAttentive Walk-Aggregating Graph Neural Network
