ExPrIS: Knowledge-Level Expectations as Priors for Object Interpretation from Sensor Data

TL;DR

This paper introduces ExPrIS, a method that combines contextual and semantic knowledge using a graph neural network to improve robotic object interpretation from sensor data, enhancing robustness and consistency.

Contribution

It proposes a novel integration of knowledge-level expectations into a GNN framework for incremental 3D scene understanding in robotics.

Findings

Improved semantic consistency in object recognition.

Enhanced robustness of scene interpretation over time.

Effective integration of external semantic knowledge.

Abstract

While deep learning has significantly advanced robotic object recognition, purely data-driven approaches often lack semantic consistency and fail to leverage valuable, pre-existing knowledge about the environment. This report presents the ExPrIS project, which addresses this challenge by investigating how knowledge-level expectations can serve as to improve object interpretation from sensor data. Our approach is based on the incremental construction of a 3D Semantic Scene Graph (3DSSG). We integrate expectations from two sources: contextual priors from past observations and semantic knowledge from external graphs like ConceptNet. These are embedded into a heterogeneous Graph Neural Network (GNN) to create an expectation-biased inference process. This method moves beyond static, frame-by-frame analysis to enhance the robustness and consistency of scene understanding over time. The report details this architecture, its evaluation, and outlines its planned integration on a mobile robotic platform.