Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics

TL;DR

This paper introduces a framework that uses large language models to explain and reconcile differences between human and robot models in assistive robotics, enabling better collaboration and correction without formal user models.

Contribution

It presents a novel model reconciliation framework leveraging LLMs for explainability and collaborative recovery in assistive robotics, without requiring formal user models.

Findings

Successful implementation on a wheelchair-based mobile manipulator

Effective explanation of model differences using LLMs

Improved human-robot collaboration through model correction

Abstract

Whenever humans and robots work together, it is essential that unexpected robot behavior can be explained to the user. Especially in applications such as shared control the user and the robot must share the same model of the objects in the world, and the actions that can be performed on these objects. In this paper, we achieve this with a so-called model reconciliation framework. We leverage a Large Language Model to predict and explain the difference between the robot's and the human's mental models, without the need of a formal mental model of the user. Furthermore, our framework aims to solve the model divergence after the explanation by allowing the human to correct the robot. We provide an implementation in an assistive robotics domain, where we conduct a set of experiments with a real wheelchair-based mobile manipulator and its digital twin.