Semantics in Robotics: Environmental Data Can't Yield Conventions of Human Behaviour

TL;DR

This paper argues that environmental data alone cannot provide semantics in robotics, which are better understood as conventions of human behavior, including labels, places, and affordances, requiring complex understanding beyond environmental data.

Contribution

It clarifies the conceptual distinction between environmental data and semantics, emphasizing that semantics involve human conventions and are not directly extractable from environmental data.

Findings

Semantics in robotics are conventions of human behavior.

Environmental data alone cannot yield true semantics.

Understanding affordances requires complex physics and object knowledge.

Abstract

The word semantics, in robotics and AI, has no canonical definition. It usually serves to denote additional data provided to autonomous agents to aid HRI. Most researchers seem, implicitly, to understand that such data cannot simply be extracted from environmental data. I try to make explicit why this is so and argue that so-called semantics are best understood as data comprised of conventions of human behaviour. This includes labels, most obviously, but also places, ontologies, and affordances. Object affordances are especially problematic because they require not only semantics that are not in the environmental data (conventions of object use) but also an understanding of physics and object combinations that would, if achieved, constitute artificial superintelligence.