Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent

TL;DR

ROSA is an AI-powered agent that enables natural language interaction with ROS-based robots, making robotic systems more accessible, safe, and adaptable for diverse users and mission scenarios.

Contribution

This paper introduces ROSA, a modular, extensible AI agent that bridges natural language interfaces with ROS, enhancing human-robot interaction and operational safety.

Findings

ROSA successfully translates natural language commands into robot actions.

ROSA demonstrates compatibility with ROS1 and ROS2 in various test environments.

Initial mock-ups show improved accessibility and safety in robot operations.

Abstract

The advancement of robotic systems has revolutionized numerous industries, yet their operation often demands specialized technical knowledge, limiting accessibility for non-expert users. This paper introduces ROSA (Robot Operating System Agent), an AI-powered agent that bridges the gap between the Robot Operating System (ROS) and natural language interfaces. By leveraging state-of-the-art language models and integrating open-source frameworks, ROSA enables operators to interact with robots using natural language, translating commands into actions and interfacing with ROS through well-defined tools. ROSA's design is modular and extensible, offering seamless integration with both ROS1 and ROS2, along with safety mechanisms like parameter validation and constraint enforcement to ensure secure, reliable operations. While ROSA is originally designed for ROS, it can be extended to work with other robotics middle-wares to maximize compatibility across missions. ROSA enhances human-robot interaction by democratizing access to complex robotic systems, empowering users of all expertise levels with multi-modal capabilities such as speech integration and visual perception. Ethical considerations are thoroughly addressed, guided by foundational principles like Asimov's Three Laws of Robotics, ensuring that AI integration promotes safety, transparency, privacy, and accountability. By making robotic technology more user-friendly and accessible, ROSA not only improves operational efficiency but also sets a new standard for responsible AI use in robotics and potentially future mission operations. This paper introduces ROSA's architecture and showcases initial mock-up operations in JPL's Mars Yard, a laboratory, and a simulation using three different robots. The core ROSA library is available as open-source.