RoomRecon: High-Quality Textured Room Layout Reconstruction on Mobile Devices

TL;DR

RoomRecon is a real-time, mobile-friendly 3D room reconstruction system that enhances texturing quality using AR-guided image capture and AI, focusing on permanent room elements for better realism.

Contribution

It introduces a two-phase texturing pipeline combining AR-guided image capture and generative AI, improving on existing methods for mobile indoor scene reconstruction.

Findings

Outperforms state-of-the-art in texturing quality

Operates efficiently on mobile devices

Receives positive user feedback in experiments

Abstract

Widespread RGB-Depth (RGB-D) sensors and advanced 3D reconstruction technologies facilitate the capture of indoor spaces, improving the fields of augmented reality (AR), virtual reality (VR), and extended reality (XR). Nevertheless, current technologies still face limitations, such as the inability to reflect minor scene changes without a complete recapture, the lack of semantic scene understanding, and various texturing challenges that affect the 3D model's visual quality. These issues affect the realism required for VR experiences and other applications such as in interior design and real estate. To address these challenges, we introduce RoomRecon, an interactive, real-time scanning and texturing pipeline for 3D room models. We propose a two-phase texturing pipeline that integrates AR-guided image capturing for texturing and generative AI models to improve texturing quality and provide better replicas of indoor spaces. Moreover, we suggest focusing only on permanent room elements such as walls, floors, and ceilings, to allow for easily customizable 3D models. We conduct experiments in a variety of indoor spaces to assess the texturing quality and speed of our method. The quantitative results and user study demonstrate that RoomRecon surpasses state-of-the-art methods in terms of texturing quality and on-device computation time.