Mononizing Binocular Videos

TL;DR

This paper introduces a novel framework to convert binocular videos into monocular videos with implicit stereo information, enabling easy distribution and high-quality restoration of the original 3D content on stereoscopic displays.

Contribution

We propose a self-supervised encoding-decoding framework with specialized modules to effectively mononize binocular videos and restore the original stereo content, addressing artifact suppression and noise resilience.

Findings

High-quality mononized videos with preserved stereo information

Effective restoration of original binocular videos from mononized versions

Positive user perception and quantitative metrics confirming method effectiveness

Abstract

This paper presents the idea ofmono-nizingbinocular videos and a frame-work to effectively realize it. Mono-nize means we purposely convert abinocular video into a regular monocular video with the stereo informationimplicitly encoded in a visual but nearly-imperceptible form. Hence, wecan impartially distribute and show the mononized video as an ordinarymonocular video. Unlike ordinary monocular videos, we can restore from itthe original binocular video and show it on a stereoscopic display. To start,we formulate an encoding-and-decoding framework with the pyramidal de-formable fusion module to exploit long-range correspondences between theleft and right views, a quantization layer to suppress the restoring artifacts,and the compression noise simulation module to resist the compressionnoise introduced by modern video codecs. Our framework is self-supervised,as we articulate our objective function with loss terms defined on the input:a monocular term for creating the mononized video, an invertibility termfor restoring the original video, and a temporal term for frame-to-framecoherence. Further, we conducted extensive experiments to evaluate ourgenerated mononized videos and restored binocular videos for diverse typesof images and 3D movies. Quantitative results on both standard metrics anduser perception studies show the effectiveness of our method.