IROSA: Interactive Robot Skill Adaptation using Natural Language

TL;DR

This paper introduces IROSA, a framework that uses large language models to adapt robot skills via natural language commands, enhancing industrial robot flexibility without fine-tuning.

Contribution

It presents a novel tool-based architecture enabling open-vocabulary skill adaptation for industrial robots using pre-trained language models without fine-tuning.

Findings

Successful adaptation of robot skills via natural language commands

Demonstrated on a 7-DoF industrial robot performing insertion tasks

Maintained safety, transparency, and interpretability during adaptation

Abstract

Foundation models have demonstrated impressive capabilities across diverse domains, while imitation learning provides principled methods for robot skill adaptation from limited data. Combining these approaches holds significant promise for direct application to robotics, yet this combination has received limited attention, particularly for industrial deployment. We present a novel framework that enables open-vocabulary skill adaptation through a tool-based architecture, maintaining a protective abstraction layer between the language model and robot hardware. Our approach leverages pre-trained LLMs to select and parameterize specific tools for adapting robot skills without requiring fine-tuning or direct model-to-robot interaction. We demonstrate the framework on a 7-DoF torque-controlled robot performing an industrial bearing ring insertion task, showing successful skill adaptation through natural language commands for speed adjustment, trajectory correction, and obstacle avoidance while maintaining safety, transparency, and interpretability.