Semantic UV mapping to improve texture inpainting for indoor scenes

TL;DR

This paper introduces a semantic UV mapping pre-processing technique that leverages scene semantics to enhance texture inpainting and 3D reconstruction accuracy in indoor scanned environments.

Contribution

It proposes a novel semantic UV mapping method that integrates semantic and visual features to improve UV unwrapping and scene segmentation for indoor scenes.

Findings

Enhanced UV mapping accuracy for indoor scenes

Improved texture inpainting after clutter removal

Simplified 3D scene reconstruction process

Abstract

This work aims to improve texture inpainting after clutter removal in scanned indoor meshes. This is achieved with a new UV mapping pre-processing step which leverages semantic information of indoor scenes to more accurately match the UV islands with the 3D representation of distinct structural elements like walls and floors. Semantic UV Mapping enriches classic UV unwrapping algorithms by not only relying on geometric features but also visual features originating from the present texture. The segmentation improves the UV mapping and simultaneously simplifies the 3D geometric reconstruction of the scene after the removal of loose objects. Each segmented element can be reconstructed separately using the boundary conditions of the adjacent elements. Because this is performed as a pre-processing step, other specialized methods for geometric and texture reconstruction can be used in the future to improve the results even further.