Commonsense Reasoning for Identifying and Understanding the Implicit Need of Help and Synthesizing Assistive Actions

TL;DR

This paper introduces an architecture for robots to recognize implicit human needs for help and to generate appropriate assistive actions using commonsense reasoning, without prior training, enhancing human-robot interaction.

Contribution

It presents a novel integrated architecture combining scene graph generation, commonsense knowledge, and sentiment analysis to detect needs and synthesize actions in HRI.

Findings

Effective detection of implicit needs using scene graphs and commonsense

Generation of assistive actions without prior learning

Evaluation on established benchmarks and human studies

Abstract

Human-Robot Interaction (HRI) is an emerging subfield of service robotics. While most existing approaches rely on explicit signals (i.e. voice, gesture) to engage, current literature is lacking solutions to address implicit user needs. In this paper, we present an architecture to (a) detect user implicit need of help and (b) generate a set of assistive actions without prior learning. Task (a) will be performed using state-of-the-art solutions for Scene Graph Generation coupled to the use of commonsense knowledge; whereas, task (b) will be performed using additional commonsense knowledge as well as a sentiment analysis on graph structure. Finally, we propose an evaluation of our solution using established benchmarks (e.g. ActionGenome dataset) along with human experiments. The main motivation of our approach is the embedding of the perception-decision-action loop in a single architecture.