Self-supervised Pre-training for Semantic Segmentation in an Indoor Scene

TL;DR

This paper introduces RegConsist, a self-supervised pre-training method for semantic segmentation in indoor scenes, leveraging multi-view consistency and contrastive learning to improve performance without extensive labeled data.

Contribution

The paper presents a novel self-supervised pre-training approach that exploits spatial and temporal consistency cues for semantic segmentation in indoor environments.

Findings

Outperforms ImageNet pre-trained models in semantic segmentation tasks.

Achieves competitive results compared to task-specific models trained on different datasets.

Demonstrates effectiveness through comprehensive ablation studies.

Abstract

The ability to endow maps of indoor scenes with semantic information is an integral part of robotic agents which perform different tasks such as target driven navigation, object search or object rearrangement. The state-of-the-art methods use Deep Convolutional Neural Networks (DCNNs) for predicting semantic segmentation of an image as useful representation for these tasks. The accuracy of semantic segmentation depends on the availability and the amount of labeled data from the target environment or the ability to bridge the domain gap between test and training environment. We propose RegConsist, a method for self-supervised pre-training of a semantic segmentation model, exploiting the ability of the agent to move and register multiple views in the novel environment. Given the spatial and temporal consistency cues used for pixel level data association, we use a variant of contrastive learning to train a DCNN model for predicting semantic segmentation from RGB views in the target environment. The proposed method outperforms models pre-trained on ImageNet and achieves competitive performance when using models that are trained for exactly the same task but on a different dataset. We also perform various ablation studies to analyze and demonstrate the efficacy of our proposed method.