Dense Scene Reconstruction from Light-Field Images Affected by Rolling Shutter

TL;DR

This paper introduces a novel method for dense depth estimation from light-field images that compensates for rolling shutter effects, utilizing a two-stage approach with synthetic data for evaluation.

Contribution

The paper proposes a new two-stage approach for dense scene reconstruction from light-field images affected by rolling shutter, including a synthetic dataset for evaluation.

Findings

Effective compensation for rolling shutter effects demonstrated

Two-stage method improves depth estimation accuracy

Synthetic dataset facilitates evaluation of rolling shutter effects

Abstract

This paper presents a dense depth estimation approach from light-field (LF) images that is able to compensate for strong rolling shutter (RS) effects. Our method estimates RS compensated views and dense RS compensated disparity maps. We present a two-stage method based on a 2D Gaussians Splatting that allows for a ``render and compare" strategy with a point cloud formulation. In the first stage, a subset of sub-aperture images is used to estimate an RS agnostic 3D shape that is related to the scene target shape ``up to a motion". In the second stage, the deformation of the 3D shape is computed by estimating an admissible camera motion. We demonstrate the effectiveness and advantages of this approach through several experiments conducted for different scenes and types of motions. Due to lack of suitable datasets for evaluation, we also present a new carefully designed synthetic dataset of RS LF images. The source code, trained models and dataset will be made publicly available at: https://github.com/ICB-Vision-AI/DenseRSLF