IM360: Large-scale Indoor Mapping with 360 Cameras

TL;DR

IM360 introduces a comprehensive 3D indoor mapping pipeline utilizing 360 cameras, spherical SfM, and neural rendering to improve large-scale indoor scene reconstruction and camera localization.

Contribution

The paper presents IM360, a novel large-scale indoor mapping approach that integrates omnidirectional images, spherical SfM, and neural rendering for enhanced accuracy.

Findings

Achieves 3.5 PSNR increase in textured mesh reconstruction

State-of-the-art camera localization on Matterport3D and Stanford2D3D

Effective handling of occlusions and textureless regions in indoor scenes

Abstract

We present a novel 3D mapping pipeline for large-scale indoor environments. To address the significant challenges in large-scale indoor scenes, such as prevalent occlusions and textureless regions, we propose IM360, a novel approach that leverages the wide field of view of omnidirectional images and integrates the spherical camera model into the Structure-from-Motion (SfM) pipeline. Our SfM utilizes dense matching features specifically designed for 360 images, demonstrating superior capability in image registration. Furthermore, with the aid of mesh-based neural rendering techniques, we introduce a texture optimization method that refines texture maps and accurately captures view-dependent properties by combining diffuse and specular components. We evaluate our pipeline on large-scale indoor scenes, demonstrating its effectiveness in real-world scenarios. In practice, IM360 demonstrates superior performance, achieving a 3.5 PSNR increase in textured mesh reconstruction. We attain state-of-the-art performance in terms of camera localization and registration on Matterport3D and Stanford2D3D.