Implicit Preference Alignment for Human Image Animation

TL;DR

This paper introduces Implicit Preference Alignment (IPA), a data-efficient framework for improving human image animation, especially hand motions, without requiring preference pair data, by leveraging implicit reward maximization.

Contribution

The paper proposes a novel, data-efficient post-training method that aligns models using implicit reward maximization, reducing the need for costly preference data in human image animation.

Findings

Effective enhancement of hand motion quality in image animation.

Significant reduction in preference data requirements.

Successful implementation of Hand-Aware Local Optimization.

Abstract

Human image animation has witnessed significant advancements, yet generating high-fidelity hand motions remains a persistent challenge due to their high degrees of freedom and motion complexity. While reinforcement learning from human feedback, particularly direct preference optimization, offers a potential solution, it necessitates the construction of strict preference pairs. However, curating such pairs for dynamic hand regions is prohibitively expensive and often impractical due to frame-wise inconsistencies. In this paper, we propose Implicit Preference Alignment (IPA), a data-efficient post-training framework that eliminates the need for paired preference data. Theoretically grounded in implicit reward maximization, IPA aligns the model by maximizing the likelihood of self-generated high-quality samples while penalizing deviations from the pretrained prior. Furthermore, we introduce a Hand-Aware Local Optimization mechanism to explicitly steer the alignment process toward hand regions. Experiments demonstrate that our method achieves effective preference optimization to enhance hand generation quality, while significantly lowering the barrier for constructing preference data. Codes are released at https://github.com/mdswyz/IPA