MotionFix: Text-Driven 3D Human Motion Editing

TL;DR

This paper introduces MotionFix, a new dataset and a diffusion-based model for text-driven 3D human motion editing, demonstrating improved editing capabilities and establishing benchmarks for evaluation.

Contribution

The paper presents a novel dataset and a diffusion model for fine-grained text-driven 3D motion editing, addressing data scarcity and editing accuracy challenges.

Findings

The diffusion model outperforms baselines trained on text-motion pairs.

New retrieval-based metrics effectively evaluate motion editing quality.

The MotionFix dataset enables better training and benchmarking.

Abstract

The focus of this paper is on 3D motion editing. Given a 3D human motion and a textual description of the desired modification, our goal is to generate an edited motion as described by the text. The key challenges include the scarcity of training data and the need to design a model that accurately edits the source motion. In this paper, we address both challenges. We propose a methodology to semi-automatically collect a dataset of triplets comprising (i) a source motion, (ii) a target motion, and (iii) an edit text, introducing the new MotionFix dataset. Access to this data allows us to train a conditional diffusion model, TMED, that takes both the source motion and the edit text as input. We develop several baselines to evaluate our model, comparing it against models trained solely on text-motion pair datasets, and demonstrate the superior performance of our model trained on triplets. We also introduce new retrieval-based metrics for motion editing, establishing a benchmark on the evaluation set of MotionFix. Our results are promising, paving the way for further research in fine-grained motion generation. Code, models, and data are available at https://motionfix.is.tue.mpg.de/ .