ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation

TL;DR

ClusterStyle introduces a prototype-based clustering framework to effectively model intra-style diversity in stylized motion generation, capturing both global and local style variations and enhancing style transfer quality.

Contribution

The paper proposes a novel clustering-based approach with prototypes to model intra-style diversity, improving stylized motion generation and style transfer over existing methods.

Findings

Outperforms state-of-the-art models in stylized motion generation

Effectively models intra-style diversity with global and local style embeddings

Enhances motion style transfer quality

Abstract

Existing stylized motion generation models have shown their remarkable ability to understand specific style information from the style motion, and insert it into the content motion. However, capturing intra-style diversity, where a single style should correspond to diverse motion variations, remains a significant challenge. In this paper, we propose a clustering-based framework, ClusterStyle, to address this limitation. Instead of learning an unstructured embedding from each style motion, we leverage a set of prototypes to effectively model diverse style patterns across motions belonging to the same style category. We consider two types of style diversity: global-level diversity among style motions of the same category, and local-level diversity within the temporal dynamics of motion sequences. These components jointly shape two structured style embedding spaces, i.e., global and local, optimized via alignment with non-learnable prototype anchors. Furthermore, we augment the pretrained text-to-motion generation model with the Stylistic Modulation Adapter (SMA) to integrate the style features. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art models in stylized motion generation and motion style transfer.