ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis

TL;DR

ChatMotion is a multimodal multi-agent framework that enhances human motion analysis by interpreting user intent, decomposing tasks, and integrating specialized modules for improved understanding and interactivity.

Contribution

It introduces a novel multi-agent system that dynamically interprets user needs and combines multiple modules for comprehensive human motion analysis.

Findings

Demonstrates high precision in motion understanding

Shows adaptability to diverse analytical tasks

Engages users effectively in motion analysis

Abstract

Advancements in Multimodal Large Language Models (MLLMs) have improved human motion understanding. However, these models remain constrained by their "instruct-only" nature, lacking interactivity and adaptability for diverse analytical perspectives. To address these challenges, we introduce ChatMotion, a multimodal multi-agent framework for human motion analysis. ChatMotion dynamically interprets user intent, decomposes complex tasks into meta-tasks, and activates specialized function modules for motion comprehension. It integrates multiple specialized modules, such as the MotionCore, to analyze human motion from various perspectives. Extensive experiments demonstrate ChatMotion's precision, adaptability, and user engagement for human motion understanding.