MagicFight: Personalized Martial Arts Combat Video Generation

TL;DR

MagicFight introduces a novel framework for generating personalized two-person martial arts combat videos, addressing challenges like identity preservation and action coherence, and provides a new dataset for this emerging task.

Contribution

It pioneers the task of personalized martial arts combat video generation and creates a new dataset using Unity to facilitate future research.

Findings

High-fidelity two-person combat videos generated with preserved identities.

Seamless, coherent action sequences achieved in generated videos.

Established a new benchmark for interactive combat video synthesis.

Abstract

Amid the surge in generic text-to-video generation, the field of personalized human video generation has witnessed notable advancements, primarily concentrated on single-person scenarios. However, to our knowledge, the domain of two-person interactions, particularly in the context of martial arts combat, remains uncharted. We identify a significant gap: existing models for single-person dancing generation prove insufficient for capturing the subtleties and complexities of two engaged fighters, resulting in challenges such as identity confusion, anomalous limbs, and action mismatches. To address this, we introduce a pioneering new task, Personalized Martial Arts Combat Video Generation. Our approach, MagicFight, is specifically crafted to overcome these hurdles. Given this pioneering task, we face a lack of appropriate datasets. Thus, we generate a bespoke dataset using the game physics engine Unity, meticulously crafting a multitude of 3D characters, martial arts moves, and scenes designed to represent the diversity of combat. MagicFight refines and adapts existing models and strategies to generate high-fidelity two-person combat videos that maintain individual identities and ensure seamless, coherent action sequences, thereby laying the groundwork for future innovations in the realm of interactive video content creation. Website: https://MingfuYAN.github.io/MagicFight/ Dataset: https://huggingface.co/datasets/MingfuYAN/KungFu-Fiesta