# Mesh-based Camera Pairs Selection and Occlusion-Aware Masking for Mesh Refinement

## TL;DR

This paper introduces a mesh refinement method that uses a novel camera pair selection strategy based on five metrics and an occlusion-aware approach to improve 3D scene reconstruction quality.

## Contribution

It proposes a new camera pair selection method leveraging the initial 3D model and introduces an occlusion-aware refinement technique for better mesh accuracy.

## Key findings

- Improved mesh refinement quality over state-of-the-art methods.
- Effective camera pair selection based on scene coverage, overlap, resolution, parallax, and symmetry.
- Robustness to occlusions enhances reconstruction accuracy.

## Abstract

Many Multi-View-Stereo algorithms extract a 3D mesh model of a scene, after fusing depth maps into a volumetric representation of the space. Due to the limited scalability of such representations, the estimated model does not capture fine details of the scene. Therefore a mesh refinement algorithm is usually applied; it improves the mesh resolution and accuracy by minimizing the photometric error induced by the 3D model into pairs of cameras. The choice of these pairs significantly affects the quality of the refinement and usually relies on sparse 3D points belonging to the surface. Instead, in this paper, to increase the quality of pairs selection, we exploit the 3D model (before the refinement) to compute five metrics: scene coverage, mutual image overlap, image resolution, camera parallax, and a new symmetry term. To improve the refinement robustness, we also propose an explicit method to manage occlusions, which may negatively affect the computation of the photometric error. The proposed method takes into account the depth of the model while computing the similarity measure and its gradient. We quantitatively and qualitatively validated our approach on publicly available datasets against state of the art reconstruction methods.