Lightweight Multiplane Images Network for Real-Time Stereoscopic Conversion from Planar Video
Shanding Diao, Yang Zhao, Yuan Chen, Zhao Zhang, Wei Jia, Ronggang, Wang

TL;DR
This paper introduces a lightweight, real-time stereoscopic conversion network using multi-plane images that balances high-quality reconstruction with efficient inference, suitable for 2K resolution virtual reality and 3D displays.
Contribution
It proposes a novel multi-plane image network with a lightweight depth-semantic branch and a simplified MPI rendering process for real-time stereoscopic conversion.
Findings
Achieves real-time 2K resolution inference.
Provides over 40x acceleration compared to SOTA TMPI.
Maintains comparable visual quality to state-of-the-art models.
Abstract
With the rapid development of stereoscopic display technologies, especially glasses-free 3D screens, and virtual reality devices, stereoscopic conversion has become an important task to address the lack of high-quality stereoscopic image and video resources. Current stereoscopic conversion algorithms typically struggle to balance reconstruction performance and inference efficiency. This paper proposes a planar video real-time stereoscopic conversion network based on multi-plane images (MPI), which consists of a detail branch for generating MPI and a depth-semantic branch for perceiving depth information. Unlike models that depend on explicit depth map inputs, the proposed method employs a lightweight depth-semantic branch to extract depth-aware features implicitly. To optimize the lightweight branch, a heavy training but light inference strategy is adopted, which involves designing a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Advanced Image Processing Techniques · Image and Video Stabilization
