3D-LMVIC: Learning-based Multi-View Image Coding with 3D Gaussian Geometric Priors

TL;DR

3D-LMVIC introduces a learning-based multi-view image compression framework utilizing 3D Gaussian Splatting for improved disparity estimation and geometric redundancy reduction, outperforming existing methods especially in wide-baseline scenarios.

Contribution

It presents a novel 3D Gaussian-based approach for multi-view image compression, addressing limitations of 2D projection methods in complex disparity settings.

Findings

Outperforms traditional and learning-based methods in compression quality.

Significantly improves disparity estimation accuracy.

Effectively reduces geometric redundancy across views.

Abstract

Existing multi-view image compression methods often rely on 2D projection-based similarities between views to estimate disparities. While effective for small disparities, such as those in stereo images, these methods struggle with the more complex disparities encountered in wide-baseline multi-camera systems, commonly found in virtual reality and autonomous driving applications. To address this limitation, we propose 3D-LMVIC, a novel learning-based multi-view image compression framework that leverages 3D Gaussian Splatting to derive geometric priors for accurate disparity estimation. Furthermore, we introduce a depth map compression model to minimize geometric redundancy across views, along with a multi-view sequence ordering strategy based on a defined distance measure between views to enhance correlations between adjacent views. Experimental results demonstrate that 3D-LMVIC achieves superior performance compared to both traditional and learning-based methods. Additionally, it significantly improves disparity estimation accuracy over existing two-view approaches.