HiFi-Inpaint: Towards High-Fidelity Reference-Based Inpainting for Generating Detail-Preserving Human-Product Images

TL;DR

HiFi-Inpaint is a new framework for high-fidelity reference-based inpainting that preserves product details in human-product images, utilizing novel attention and loss mechanisms, and trained on a large curated dataset.

Contribution

It introduces Shared Enhancement Attention and Detail-Aware Loss for improved detail preservation, along with a new dataset HP-Image-40K for training.

Findings

Achieves state-of-the-art detail preservation in human-product image inpainting.

Outperforms existing methods on the HP-Image-40K dataset.

Demonstrates effective guidance of product details through proposed mechanisms.

Abstract

Human-product images, which showcase the integration of humans and products, play a vital role in advertising, e-commerce, and digital marketing. The essential challenge of generating such images lies in ensuring the high-fidelity preservation of product details. Among existing paradigms, reference-based inpainting offers a targeted solution by leveraging product reference images to guide the inpainting process. However, limitations remain in three key aspects: the lack of diverse large-scale training data, the struggle of current models to focus on product detail preservation, and the inability of coarse supervision for achieving precise guidance. To address these issues, we propose HiFi-Inpaint, a novel high-fidelity reference-based inpainting framework tailored for generating human-product images. HiFi-Inpaint introduces Shared Enhancement Attention (SEA) to refine fine-grained product features and Detail-Aware Loss (DAL) to enforce precise pixel-level supervision using high-frequency maps. Additionally, we construct a new dataset, HP-Image-40K, with samples curated from self-synthesis data and processed with automatic filtering. Experimental results show that HiFi-Inpaint achieves state-of-the-art performance, delivering detail-preserving human-product images.