Topological Semantic Mapping by Consolidation of Deep Visual Features

TL;DR

This paper presents a topological semantic mapping approach that consolidates deep visual features from CNNs to recognize diverse semantic properties and improve spatial understanding in indoor environments.

Contribution

It introduces a novel method that consolidates CNN-extracted features for flexible semantic recognition and topological localization in robotic mapping.

Findings

Effective consolidation of visual features for semantic recognition

Improved object and place category identification

Accurate topological localization of images

Abstract

Many works in the recent literature introduce semantic mapping methods that use CNNs (Convolutional Neural Networks) to recognize semantic properties in images. The types of properties (eg.: room size, place category, and objects) and their classes (eg.: kitchen and bathroom, for place category) are usually predefined and restricted to a specific task. Thus, all the visual data acquired and processed during the construction of the maps are lost and only the recognized semantic properties remain on the maps. In contrast, this work introduces a topological semantic mapping method that uses deep visual features extracted by a CNN (GoogLeNet), from 2D images captured in multiple views of the environment as the robot operates, to create, through averages, consolidated representations of the visual features acquired in the regions covered by each topological node. These representations allow flexible recognition of semantic properties of the regions and use in other visual tasks. Experiments with a real-world indoor dataset showed that the method is able to consolidate the visual features of regions and use them to recognize objects and place categories as semantic properties, and to indicate the topological location of images, with very promising results.