M2HRI: An LLM-Driven Multimodal Multi-Agent Framework for Personalized Human-Robot Interaction

TL;DR

This paper presents M2HRI, a multimodal multi-agent framework using large language models that assigns distinct personalities and memory to robots, improving personalization, coordination, and interaction quality in human-robot interactions.

Contribution

It introduces a novel LLM-driven multi-agent framework that incorporates individuality and structured coordination for enhanced human-robot interaction.

Findings

LLM-driven personality traits are significantly distinguishable.

Long-term memory enhances personalization and preference awareness.

Centralized coordination reduces overlap and improves interaction quality.

Abstract

Multi-robot systems hold significant promise for social environments such as homes and hospitals, yet existing multi-robot works treat robots as functionally identical, overlooking how robots individual identity shape user perception and how coordination shapes multi-robot behavior when such individuality is present. To address this, we introduce M2HRI, a multimodal multi-agent framework built on large language models that equips each robot with distinct personality and long-term memory, alongside a coordination mechanism conditioned on these differences. In a controlled user study (n = 105) in a multi-agent human-robot interaction (HRI) scenario, we find that LLM-driven personality traits are significantly distinguishable and enhance interaction quality, long-term memory improves personalization and preference awareness, and centralized coordination significantly reduces overlap while improving overall interaction quality. Together, these results demonstrate that both agent individuality and structured coordination are essential for coherent and socially appropriate multi-agent HRI. Project website and code are available at https://project-m2hri.github.io/.