Customize Your Own Paired Data via Few-shot Way

TL;DR

This paper introduces a few-shot learning framework for image editing that allows users to customize effects with minimal paired data, overcoming limitations of existing supervised and unsupervised methods.

Contribution

It proposes a novel few-shot learning approach using directional transformations and diffusion models to enable flexible image editing with limited data.

Findings

Effective in various image editing scenarios

Requires only a few image pairs for customization

Outperforms some existing methods in flexibility and data efficiency

Abstract

Existing solutions to image editing tasks suffer from several issues. Though achieving remarkably satisfying generated results, some supervised methods require huge amounts of paired training data, which greatly limits their usages. The other unsupervised methods take full advantage of large-scale pre-trained priors, thus being strictly restricted to the domains where the priors are trained on and behaving badly in out-of-distribution cases. The task we focus on is how to enable the users to customize their desired effects through only few image pairs. In our proposed framework, a novel few-shot learning mechanism based on the directional transformations among samples is introduced and expands the learnable space exponentially. Adopting a diffusion model pipeline, we redesign the condition calculating modules in our model and apply several technical improvements. Experimental results demonstrate the capabilities of our method in various cases.