Adversarial Open Domain Adaption Framework (AODA): Sketch-to-Photo Synthesis

TL;DR

This paper introduces an adversarial open domain adaptation framework for sketch-to-photo synthesis, effectively generating realistic images from sketches despite domain gaps and lack of supervised data, outperforming current methods.

Contribution

The paper proposes a novel open domain sampling and optimization approach that improves sketch-to-photo synthesis across diverse categories without supervision.

Findings

Outperforms current methods on Scribble and SketchyCOCO datasets.

Generates accurate color, substance, and structural layout for open-domain sketches.

Effectively handles domain gaps between synthetic and genuine sketches.

Abstract

This paper aims to demonstrate the efficiency of the Adversarial Open Domain Adaption framework for sketch-to-photo synthesis. The unsupervised open domain adaption for generating realistic photos from a hand-drawn sketch is challenging as there is no such sketch of that class for training data. The absence of learning supervision and the huge domain gap between both the freehand drawing and picture domains make it hard. We present an approach that learns both sketch-to-photo and photo-to-sketch generation to synthesise the missing freehand drawings from pictures. Due to the domain gap between synthetic sketches and genuine ones, the generator trained on false drawings may produce unsatisfactory results when dealing with drawings of lacking classes. To address this problem, we offer a simple but effective open-domain sampling and optimization method that tricks the generator into considering false drawings as genuine. Our approach generalises the learnt sketch-to-photo and photo-to-sketch mappings from in-domain input to open-domain categories. On the Scribble and SketchyCOCO datasets, we compared our technique to the most current competing methods. For many types of open-domain drawings, our model outperforms impressive results in synthesising accurate colour, substance, and retaining the structural layout.