Processing Natural Language About Ongoing Actions

TL;DR

This paper extends a natural language understanding system for autonomous control to handle ongoing actions, interruptions, and overrides using X-nets, demonstrated on robot motion control.

Contribution

Introduces X-nets for managing ongoing actions in natural language commands, enabling asynchronous control and handling interruptions in autonomous systems.

Findings

Supports standard semantics like stop, continue, override

Enables asynchronous processing of language requests

Applicable to various domains beyond robot motion

Abstract

Actions may not proceed as planned; they may be interrupted, resumed or overridden. This is a challenge to handle in a natural language understanding system. We describe extensions to an existing implementation for the control of autonomous systems by natural language, to enable such systems to handle incoming language requests regarding actions. Language Communication with Autonomous Systems (LCAS) has been extended with support for X-nets, parameterized executable schemas representing actions. X-nets enable the system to control actions at a desired level of granularity, while providing a mechanism for language requests to be processed asynchronously. Standard semantics supported include requests to stop, continue, or override the existing action. The specific domain demonstrated is the control of motion of a simulated robot, but the approach is general, and could be applied to other domains.