Low-Latency Immersive 6D Televisualization with Spherical Rendering

TL;DR

This paper introduces a real-time stereo scene capture and VR visualization method using spherical rendering, enabling low-latency head movement control during teleoperation with improved performance over existing systems.

Contribution

The authors propose a decoupled spherical rendering approach for dynamic scenes in VR teleoperation, reducing latency and distortion compared to scene reconstruction methods.

Findings

Outperforms existing visualization methods in latency and distortion

Lab experiments demonstrate improved re-rendering speed

User study shows enhanced user experience and reduced motion sickness

Abstract

We present a method for real-time stereo scene capture and remote VR visualization that allows a human operator to freely move their head and thus intuitively control their perspective during teleoperation. The stereo camera is mounted on a 6D robotic arm, which follows the operator's head pose. Existing VR teleoperation systems either induce high latencies on head movements, leading to motion sickness, or use scene reconstruction methods to allow re-rendering of the scene from different perspectives, which cannot handle dynamic scenes effectively. Instead, we present a decoupled approach which renders captured camera images as spheres, assuming constant distance. This allows very fast re-rendering on head pose changes while keeping the resulting temporary distortions during head translations small. We present qualitative examples, quantitative results in the form of lab experiments and a small user study, showing that our method outperforms other visualization methods.