Teaching Perception

TL;DR

This paper presents a robotic perception system that adapts its sensing behavior based on verbal instructions, using a natural language symbolic reasoning system to guide perception in real-world scenarios.

Contribution

It introduces a novel approach where a robot's perception policy is dynamically set through natural language commands, integrating symbolic reasoning with robotic perception.

Findings

System successfully guided perception in physical robot scenarios

Demonstrated flexible perception policy adaptation via verbal instructions

Discussed the structure of the natural language reasoning system

Abstract

The visual world is very rich and generally too complex to perceive in its entirety. Yet only certain features are typically required to adequately perform some task in a given situation. Rather than hardwire-in decisions about when and what to sense, this paper describes a robotic system whose behavioral policy can be set by verbal instructions it receives. These capabilities are demonstrated in an associated video showing the fully implemented system guiding the perception of a physical robot in simple scenario. The structure and functioning of the underlying natural language based symbolic reasoning system is also discussed.