PanoMixSwap Panorama Mixing via Structural Swapping for Indoor Scene Understanding

TL;DR

PanoMixSwap is a novel data augmentation technique for indoor panoramic images that combines background, furniture, and layout from different images to enhance dataset diversity and improve scene understanding tasks.

Contribution

The paper introduces PanoMixSwap, a new method for mixing parts of indoor panoramic images to generate diverse training data for better scene understanding.

Findings

Improves semantic segmentation accuracy on indoor panoramas.

Enhances layout estimation performance.

State-of-the-art results achieved with augmented data.

Abstract

The volume and diversity of training data are critical for modern deep learningbased methods. Compared to the massive amount of labeled perspective images, 360 panoramic images fall short in both volume and diversity. In this paper, we propose PanoMixSwap, a novel data augmentation technique specifically designed for indoor panoramic images. PanoMixSwap explicitly mixes various background styles, foreground furniture, and room layouts from the existing indoor panorama datasets and generates a diverse set of new panoramic images to enrich the datasets. We first decompose each panoramic image into its constituent parts: background style, foreground furniture, and room layout. Then, we generate an augmented image by mixing these three parts from three different images, such as the foreground furniture from one image, the background style from another image, and the room structure from the third image. Our method yields high diversity since there is a cubical increase in image combinations. We also evaluate the effectiveness of PanoMixSwap on two indoor scene understanding tasks: semantic segmentation and layout estimation. Our experiments demonstrate that state-of-the-art methods trained with PanoMixSwap outperform their original setting on both tasks consistently.