Leveraging Computation of Expectation Models for Commonsense Affordance Estimation on 3D Scene Graphs

TL;DR

This paper introduces a novel framework that uses Graph Convolutional Networks to analyze 3D scene graphs for estimating object affordances, aiding robotic task planning in urban environments.

Contribution

It develops a correlation-based model for learning affordances from sparse 3D scene graph data, enabling robots to reason about object utility with human-like commonsense.

Findings

Successfully validated in real-world indoor environments.

Demonstrated ability to infer object affordances comparable to human reasoning.

Enhanced robotic task planning through improved affordance estimation.

Abstract

This article studies the commonsense object affordance concept for enabling close-to-human task planning and task optimization of embodied robotic agents in urban environments. The focus of the object affordance is on reasoning how to effectively identify object's inherent utility during the task execution, which in this work is enabled through the analysis of contextual relations of sparse information of 3D scene graphs. The proposed framework develops a Correlation Information (CECI) model to learn probability distributions using a Graph Convolutional Network, allowing to extract the commonsense affordance for individual members of a semantic class. The overall framework was experimentally validated in a real-world indoor environment, showcasing the ability of the method to level with human commonsense. For a video of the article, showcasing the experimental demonstration, please refer to the following link: https://youtu.be/BDCMVx2GiQE