PIV3CAMS: a multi-camera dataset for multiple computer vision problems and its application to novel view-point synthesis

TL;DR

This paper introduces PIV3CAMS, a multi-camera dataset with paired images and videos from three different cameras, designed to advance multiple computer vision tasks including view synthesis, and demonstrates the importance of depth information in view generation.

Contribution

The paper presents a new multi-camera dataset, PIV3CAMS, and explores depth-based methods for novel view synthesis, providing valuable data and insights for multi-view computer vision research.

Findings

Depth information significantly improves view synthesis with small camera shifts.

The dataset enables diverse applications like image enhancement and view interpolation.

Depth-integrated models outperform traditional approaches in certain scenarios.

Abstract

The modern approaches for computer vision tasks significantly rely on machine learning, which requires a large number of quality images. While there is a plethora of image datasets with a single type of images, there is a lack of datasets collected from multiple cameras. In this thesis, we introduce Paired Image and Video data from three CAMeraS, namely PIV3CAMS, aimed at multiple computer vision tasks. The PIV3CAMS dataset consists of 8385 pairs of images and 82 pairs of videos taken from three different cameras: Canon D5 Mark IV, Huawei P20, and ZED stereo camera. The dataset includes various indoor and outdoor scenes from different locations in Zurich (Switzerland) and Cheonan (South Korea). Some of the computer vision applications that can benefit from the PIV3CAMS dataset are image/video enhancement, view interpolation, image matching, and much more. We provide a careful explanation of the data collection process and detailed analysis of the data. The second part of this thesis studies the usage of depth information in the view synthesizing task. In addition to the regeneration of a current state-of-the-art algorithm, we investigate several proposed alternative models that integrate depth information geometrically. Through extensive experiments, we show that the effect of depth is crucial in small view changes. Finally, we apply our model to the introduced PIV3CAMS dataset to synthesize novel target views as an example application of PIV3CAMS.