Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience

TL;DR

This paper introduces Den-SOFT, a dense, large-space light field dataset captured with a multi-camera system, aimed at advancing 3D scene reconstruction and immersive VR/AR experiences.

Contribution

The authors developed a high-density, large-scale light field dataset using a custom multi-camera system, filling a gap in existing datasets for 3D reconstruction and immersive VR applications.

Findings

Validated dataset effectiveness on popular 3D reconstruction algorithms

Demonstrated integration of 3D reconstructions into Unity for VR

Captured large outdoor scenes with high resolution and density

Abstract

We have built a custom mobile multi-camera large-space dense light field capture system, which provides a series of high-quality and sufficiently dense light field images for various scenarios. Our aim is to contribute to the development of popular 3D scene reconstruction algorithms such as IBRnet, NeRF, and 3D Gaussian splitting. More importantly, the collected dataset, which is much denser than existing datasets, may also inspire space-oriented light field reconstruction, which is potentially different from object-centric 3D reconstruction, for immersive VR/AR experiences. We utilized a total of 40 GoPro 10 cameras, capturing images of 5k resolution. The number of photos captured for each scene is no less than 1000, and the average density (view number within a unit sphere) is 134.68. It is also worth noting that our system is capable of efficiently capturing large outdoor scenes. Addressing the current lack of large-space and dense light field datasets, we made efforts to include elements such as sky, reflections, lights and shadows that are of interest to researchers in the field of 3D reconstruction during the data capture process. Finally, we validated the effectiveness of our provided dataset on three popular algorithms and also integrated the reconstructed 3DGS results into the Unity engine, demonstrating the potential of utilizing our datasets to enhance the realism of virtual reality (VR) and create feasible interactive spaces. The dataset is available at our project website.