Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling

TL;DR

This paper introduces a multi-condition guided framework for character image animation that improves stability and decoupling in complex scenes with multiple characters, using novel input modules and a new benchmark.

Contribution

The paper proposes a novel framework with input modules for implicit decoupling, and introduces a new benchmark for multi-character image animation evaluation.

Findings

Enhanced stability in complex backgrounds.

Improved separation of multiple characters and occluded parts.

Outperforms existing methods on the new benchmark.

Abstract

Controllable character image animation has a wide range of applications. Although existing studies have consistently improved performance, challenges persist in the field of character image animation, particularly concerning stability in complex backgrounds and tasks involving multiple characters. To address these challenges, we propose a novel multi-condition guided framework for character image animation, employing several well-designed input modules to enhance the implicit decoupling capability of the model. First, the optical flow guider calculates the background optical flow map as guidance information, which enables the model to implicitly learn to decouple the background motion into background constants and background momentum during training, and generate a stable background by setting zero background momentum during inference. Second, the depth order guider calculates the order map of the characters, which transforms the depth information into the positional information of multiple characters. This facilitates the implicit learning of decoupling different characters, especially in accurately separating the occluded body parts of multiple characters. Third, the reference pose map is input to enhance the ability to decouple character texture and pose information in the reference image. Furthermore, to fill the gap of fair evaluation of multi-character image animation, we propose a new benchmark comprising about 4,000 frames. Extensive qualitative and quantitative evaluations demonstrate that our method excels in generating high-quality character animations, especially in scenarios of complex backgrounds and multiple characters.