T-SVG: Text-Driven Stereoscopic Video Generation

TL;DR

T-SVG introduces a zero-shot, text-driven system for generating stereoscopic videos by transforming text prompts into 3D point clouds and rendering dual perspectives, simplifying the creation of immersive 3D content.

Contribution

This paper presents a novel, training-free approach that converts text prompts into stereoscopic videos using point cloud transformations, advancing the accessibility and efficiency of 3D video creation.

Findings

Enables zero-shot stereoscopic video generation from text prompts.

Achieves natural depth perception through subtle parallax rendering.

Offers a flexible, model-agnostic pipeline that requires no retraining.

Abstract

The advent of stereoscopic videos has opened new horizons in multimedia, particularly in extended reality (XR) and virtual reality (VR) applications, where immersive content captivates audiences across various platforms. Despite its growing popularity, producing stereoscopic videos remains challenging due to the technical complexities involved in generating stereo parallax. This refers to the positional differences of objects viewed from two distinct perspectives and is crucial for creating depth perception. This complex process poses significant challenges for creators aiming to deliver convincing and engaging presentations. To address these challenges, this paper introduces the Text-driven Stereoscopic Video Generation (T-SVG) system. This innovative, model-agnostic, zero-shot approach streamlines video generation by using text prompts to create reference videos. These videos are transformed into 3D point cloud sequences, which are rendered from two perspectives with subtle parallax differences, achieving a natural stereoscopic effect. T-SVG represents a significant advancement in stereoscopic content creation by integrating state-of-the-art, training-free techniques in text-to-video generation, depth estimation, and video inpainting. Its flexible architecture ensures high efficiency and user-friendliness, allowing seamless updates with newer models without retraining. By simplifying the production pipeline, T-SVG makes stereoscopic video generation accessible to a broader audience, demonstrating its potential to revolutionize the field.