TL;DR
This paper introduces SSH, a self-supervised framework for image harmonization that learns from natural images without annotations, improving compositing quality by matching appearance attributes.
Contribution
The paper presents a novel self-supervised approach for image harmonization that does not require annotated datasets, utilizing a dual data augmentation strategy and a new representation fusion perspective.
Findings
Outperforms state-of-the-art methods in metrics and visual quality
Uses a new dataset created by experts for benchmarking
Demonstrates effectiveness of self-supervised learning in image harmonization
Abstract
Image harmonization aims to improve the quality of image compositing by matching the "appearance" (\eg, color tone, brightness and contrast) between foreground and background images. However, collecting large-scale annotated datasets for this task requires complex professional retouching. Instead, we propose a novel Self-Supervised Harmonization framework (SSH) that can be trained using just "free" natural images without being edited. We reformulate the image harmonization problem from a representation fusion perspective, which separately processes the foreground and background examples, to address the background occlusion issue. This framework design allows for a dual data augmentation method, where diverse [foreground, background, pseudo GT] triplets can be generated by cropping an image with perturbations using 3D color lookup tables (LUTs). In addition, we build a real-world…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
