Towards Abstract Relational Learning in Human Robot Interaction

TL;DR

This paper proposes a method for robots to learn and generalize semantic relations from human interaction by combining perception and language, enabling more adaptive and context-aware understanding.

Contribution

It introduces an integrated approach that combines visual perception, natural language, and inductive reasoning to build and enrich semantic models in robots.

Findings

Robots can infer logical rules from combined perception and language data.

Semantic relations are generalized through inductive reasoning.

Enhanced human-robot interaction via inferred knowledge.

Abstract

Humans have a rich representation of the entities in their environment. Entities are described by their attributes, and entities that share attributes are often semantically related. For example, if two books have "Natural Language Processing" as the value of their `title' attribute, we can expect that their `topic' attribute will also be equal, namely, "NLP". Humans tend to generalize such observations, and infer sufficient conditions under which the `topic' attribute of any entity is "NLP". If robots need to interact successfully with humans, they need to represent entities, attributes, and generalizations in a similar way. This ends in a contextualized cognitive agent that can adapt its understanding, where context provides sufficient conditions for a correct understanding. In this work, we address the problem of how to obtain these representations through human-robot interaction. We integrate visual perception and natural language input to incrementally build a semantic model of the world, and then use inductive reasoning to infer logical rules that capture generic semantic relations, true in this model. These relations can be used to enrich the human-robot interaction, to populate a knowledge base with inferred facts, or to remove uncertainty in the robot's sensory inputs.