Challenges in Grounding Language in the Real World

TL;DR

This paper discusses the challenges of grounding natural language in real-world robotic interactions and proposes an integrated approach combining cognitive agents with large language models.

Contribution

It introduces a novel framework that merges interactive task learning in robots with large language models to improve language grounding in physical environments.

Findings

Identifies key challenges in language grounding for robots

Proposes an integrated cognitive and linguistic system

Outlines initial implementation steps

Abstract

A long-term goal of Artificial Intelligence is to build a language understanding system that allows a human to collaborate with a physical robot using language that is natural to the human. In this paper we highlight some of the challenges in doing this, and propose a solution that integrates the abilities of a cognitive agent capable of interactive task learning in a physical robot with the linguistic abilities of a large language model. We also point the way to an initial implementation of this approach.