InfiniMotion: Mamba Boosts Memory in Transformer for Arbitrary Long Motion Generation

TL;DR

InfiniMotion introduces a novel Motion Memory Transformer with Bidirectional Mamba Memory, enabling efficient and continuous generation of arbitrarily long human motion sequences, significantly outperforming previous methods in quality and length.

Contribution

The paper presents a new autoregressive framework with enhanced memory for long motion generation, capable of producing a 1-hour continuous motion sequence.

Findings

Generated a 1-hour human motion sequence with 80,000 frames.

Achieved over 30% improvement in FID over previous methods.

Produced sequences six times longer than prior state-of-the-art.

Abstract

Text-to-motion generation holds potential for film, gaming, and robotics, yet current methods often prioritize short motion generation, making it challenging to produce long motion sequences effectively: (1) Current methods struggle to handle long motion sequences as a single input due to prohibitively high computational cost; (2) Breaking down the generation of long motion sequences into shorter segments can result in inconsistent transitions and requires interpolation or inpainting, which lacks entire sequence modeling. To solve these challenges, we propose InfiniMotion, a method that generates continuous motion sequences of arbitrary length within an autoregressive framework. We highlight its groundbreaking capability by generating a continuous 1-hour human motion with around 80,000 frames. Specifically, we introduce the Motion Memory Transformer with Bidirectional Mamba Memory, enhancing the transformer's memory to process long motion sequences effectively without overwhelming computational resources. Notably our method achieves over 30% improvement in FID and 6 times longer demonstration compared to previous state-of-the-art methods, showcasing significant advancements in long motion generation. See project webpage: https://steve-zeyu-zhang.github.io/InfiniMotion/