Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction

TL;DR

This paper introduces ST-MoE, a novel model that captures complex spatio-temporal dependencies in multi-person motion prediction more accurately and efficiently, reducing computational costs and outperforming existing methods.

Contribution

The paper proposes the Spatiotemporal-Untrammelled Mixture of Experts (ST-MoE), integrating diverse experts and bidirectional Mamba mechanisms to enhance modeling flexibility and efficiency in human motion prediction.

Findings

Outperforms state-of-the-art accuracy on four datasets.

Reduces model parameters by 41.38%.

Achieves 3.6x faster training speed.

Abstract

Comprehensively and flexibly capturing the complex spatio-temporal dependencies of human motion is critical for multi-person motion prediction. Existing methods grapple with two primary limitations: i) Inflexible spatiotemporal representation due to reliance on positional encodings for capturing spatiotemporal information. ii) High computational costs stemming from the quadratic time complexity of conventional attention mechanisms. To overcome these limitations, we propose the Spatiotemporal-Untrammelled Mixture of Experts (ST-MoE), which flexibly explores complex spatio-temporal dependencies in human motion and significantly reduces computational cost. To adaptively mine complex spatio-temporal patterns from human motion, our model incorporates four distinct types of spatiotemporal experts, each specializing in capturing different spatial or temporal dependencies. To reduce the potential computational overhead while integrating multiple experts, we introduce bidirectional spatiotemporal Mamba as experts, each sharing bidirectional temporal and spatial Mamba in distinct combinations to achieve model efficiency and parameter economy. Extensive experiments on four multi-person benchmark datasets demonstrate that our approach not only outperforms state-of-art in accuracy but also reduces model parameter by 41.38% and achieves a 3.6x speedup in training. The code is available at https://github.com/alanyz106/ST-MoE.