DIGITOUR: Automatic Digital Tours for Real-Estate Properties

TL;DR

This paper introduces an automated end-to-end pipeline for creating virtual tours of real-estate properties using equirectangular images, significantly reducing manual effort through novel tagging and detection methods.

Contribution

The paper presents a novel HSV-based tagging scheme and an integrated detection and linking pipeline for automatic virtual tour generation from 360-degree images.

Findings

Effective tag detection using YOLOv5 architecture.

Accurate digit recognition with custom MobileNet.

Successful linking of images based on detected tags.

Abstract

A virtual or digital tour is a form of virtual reality technology which allows a user to experience a specific location remotely. Currently, these virtual tours are created by following a 2-step strategy. First, a photographer clicks a 360 degree equirectangular image; then, a team of annotators manually links these images for the "walkthrough" user experience. The major challenge in the mass adoption of virtual tours is the time and cost involved in manual annotation/linking of images. Therefore, this paper presents an end-to-end pipeline to automate the generation of 3D virtual tours using equirectangular images for real-estate properties. We propose a novel HSV-based coloring scheme for paper tags that need to be placed at different locations before clicking the equirectangular images using 360 degree cameras. These tags have two characteristics: i) they are numbered to help the photographer for placement of tags in sequence and; ii) bi-colored, which allows better learning of tag detection (using YOLOv5 architecture) in an image and digit recognition (using custom MobileNet architecture) tasks. Finally, we link/connect all the equirectangular images based on detected tags. We show the efficiency of the proposed pipeline on a real-world equirectangular image dataset collected from the Housing.com database.