3D Reconstruction of Shoes for Augmented Reality

TL;DR

This paper presents a mobile-based 3D shoe reconstruction framework using Gaussian Splatting and AR to improve online shopping experiences, achieving high-quality models and segmentation accuracy.

Contribution

It introduces a novel 3D modeling method for shoes from 2D images combined with AR, and provides a new dataset for shoe segmentation.

Findings

Achieved an average PSNR of 32 for 3D models.

Created a dataset with 3120 images and 0.95 IoU score.

Enabled immersive AR interactions for online shopping.

Abstract

This paper introduces a mobile-based solution that enhances online shoe shopping through 3D modeling and Augmented Reality (AR), leveraging the efficiency of 3D Gaussian Splatting. Addressing the limitations of static 2D images, the framework generates realistic 3D shoe models from 2D images, achieving an average Peak Signal-to-Noise Ratio (PSNR) of 32, and enables immersive AR interactions via smartphones. A custom shoe segmentation dataset of 3120 images was created, with the best-performing segmentation model achieving an Intersection over Union (IoU) score of 0.95. This paper demonstrates the potential of 3D modeling and AR to revolutionize online shopping by offering realistic virtual interactions, with applicability across broader fashion categories.