ViVo: A Dataset for Volumetric Video Reconstruction and Compression

TL;DR

ViVo is a comprehensive volumetric video dataset that includes diverse, realistic content with multi-view RGB, depth, masks, and point clouds, designed to advance reconstruction and compression research.

Contribution

The paper introduces ViVo, a novel dataset with diverse, realistic volumetric video content, and benchmarks state-of-the-art methods to highlight current limitations.

Findings

Existing datasets lack diversity in semantic and low-level features.

Benchmarking reveals challenges and limitations of current reconstruction and compression methods.

ViVo dataset exposes the need for more effective algorithms in volumetric video processing.

Abstract

As research on neural volumetric video reconstruction and compression flourishes, there is a need for diverse and realistic datasets, which can be used to develop and validate reconstruction and compression models. However, existing volumetric video datasets lack diverse content in terms of both semantic and low-level features that are commonly present in real-world production pipelines. In this context, we propose a new dataset, ViVo, for VolumetrIc VideO reconstruction and compression. The dataset is faithful to real-world volumetric video production and is the first dataset to extend the definition of diversity to include both human-centric characteristics (skin, hair, etc.) and dynamic visual phenomena (transparent, reflective, liquid, etc.). Each video sequence in this database contains raw data including fourteen multi-view RGB and depth video pairs, synchronized at 30FPS with per-frame calibration and audio data, and their associated 2-D foreground masks and 3-D point clouds. To demonstrate the use of this database, we have benchmarked three state-of-the-art (SotA) 3-D reconstruction methods and two volumetric video compression algorithms. The obtained results evidence the challenging nature of the proposed dataset and the limitations of existing datasets for both volumetric video reconstruction and compression tasks, highlighting the need to develop more effective algorithms for these applications. The database and the associated results are available at https://vivo-bvicr.github.io/