MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation

TL;DR

MotionShot is a training-free framework that enables high-fidelity, coherent motion transfer across arbitrary objects in text-to-video generation by combining semantic matching and shape retargeting.

Contribution

It introduces a novel, training-free method for fine-grained motion transfer that handles significant appearance and structure differences.

Findings

Effective motion transfer across diverse objects

Preserves appearance coherence during transfer

Demonstrates superior performance in experiments

Abstract

Existing text-to-video methods struggle to transfer motion smoothly from a reference object to a target object with significant differences in appearance or structure between them. To address this challenge, we introduce MotionShot, a training-free framework capable of parsing reference-target correspondences in a fine-grained manner, thereby achieving high-fidelity motion transfer while preserving coherence in appearance. To be specific, MotionShot first performs semantic feature matching to ensure high-level alignments between the reference and target objects. It then further establishes low-level morphological alignments through reference-to-target shape retargeting. By encoding motion with temporal attention, our MotionShot can coherently transfer motion across objects, even in the presence of significant appearance and structure disparities, demonstrated by extensive experiments. The project page is available at: https://motionshot.github.io/.