Interactive multiview video system with non-complex navigation at the decoder

TL;DR

This paper introduces a novel multiview video system that shifts the complexity from the decoder to the encoder, enabling smooth, interactive view switching with optimized transmission and storage, suitable for classical decoders.

Contribution

It proposes a new approach that anticipates user navigation, reducing decoder complexity by sending auxiliary information, and employs optimization based on user behavior modeling.

Findings

System enables smooth, interactive multiview navigation.

Optimized auxiliary data reduces transmission and storage costs.

Validated through experiments as a practical solution for classical decoders.

Abstract

Multiview video with interactive and smooth view switching at the receiver is a challenging application with several issues in terms of effective use of storage and bandwidth resources, reactivity of the system, quality of the viewing experience and system complexity. The classical decoding system for generating virtual views first projects a reference or encoded frame to a given viewpoint and then fills in the holes due to potential occlusions. This last step still constitutes a complex operation with specific software or hardware at the receiver and requires a certain quantity of information from the neighboring frames for insuring consistency between the virtual images. In this work we propose a new approach that shifts most of the burden due to interactivity from the decoder to the encoder, by anticipating the navigation of the decoder and sending auxiliary information that guarantees temporal and interview consistency. This leads to an additional cost in terms of transmission rate and storage, which we minimize by using optimization techniques based on the user behavior modeling. We show by experiments that the proposed system represents a valid solution for interactive multiview systems with classical decoders.