AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation

TL;DR

AnchorCrafter is a diffusion-based system that generates high-quality, controllable human-object interaction videos for product promotion, addressing key challenges in visual fidelity and interaction realism in e-commerce advertising.

Contribution

The paper introduces novel HOI-appearance perception and HOI-motion injection techniques to improve human-object interaction video generation.

Findings

Object appearance preservation improved by 7.5%.

Object localization accuracy doubled.

Outperforms existing methods in motion consistency.

Abstract

The generation of anchor-style product promotion videos presents promising opportunities in e-commerce, advertising, and consumer engagement. Despite advancements in pose-guided human video generation, creating product promotion videos remains challenging. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Extensive experiments show that our system improves object appearance preservation by 7.5\% and doubles the object localization accuracy compared to existing state-of-the-art approaches. It also outperforms existing approaches in maintaining human motion consistency and high-quality video generation. Project page including data, code, and Huggingface demo: https://github.com/cangcz/AnchorCrafter.