MistyPilot: An Agentic Fast-Slow Thinking LLM Framework for Misty Social Robots

TL;DR

MistyPilot is a novel LLM-based framework for social robots that combines fast-slow thinking, autonomous tool management, and social intelligence to improve task execution and user interaction.

Contribution

It introduces MistyPilot, integrating a dual-agent system with a fast-slow thinking paradigm, and provides comprehensive benchmarks for evaluation.

Findings

High routing correctness and task completeness

Efficient fast-slow thinking retrieval

Effective emotion and social alignment

Abstract

With the availability of open APIs in social robots, it has become easier to customize general-purpose tools to meet users' needs. However, interpreting high-level user instructions, selecting and configuring appropriate tools, and executing them reliably remain challenging for users without programming experience. To address these challenges, we introduce MistyPilot, an agentic LLM-driven framework for autonomous tool selection, orchestration, and parameter configuration. MistyPilot comprises two core components: a Physically Interactive Agent (PIA) and a Socially Intelligent Agent (SIA). The PIA enables robust sensor-triggered and tool-driven task execution, while the SIA generates socially intelligent and emotionally aligned dialogue. MistyPilot further integrates a fast-slow thinking paradigm to capture user preferences, reduce latency, and improve task efficiency. To comprehensively evaluate MistyPilot, we contribute five benchmark datasets. Extensive experiments demonstrate the effectiveness of our framework in routing correctness, task completeness, fast-slow thinking retrieval efficiency, tool scalability,and emotion alignment. All code, datasets, and experimental videos will be made publicly available on the project webpage.