One-Shot Synthesis of Images and Segmentation Masks

TL;DR

This paper introduces OSMIS, a novel one-shot GAN model that synthesizes diverse, high-fidelity images and segmentation masks from a single example without pre-training data, advancing image-mask synthesis in restricted domains.

Contribution

OSMIS is the first model to enable high-quality, diverse image and mask synthesis in a one-shot setting without pre-training, outperforming existing single-image GANs.

Findings

OSMIS produces high-fidelity, diverse images and masks from a single example.

It outperforms state-of-the-art single-image GANs in synthesis quality.

It effectively enhances one-shot segmentation through data augmentation.

Abstract

Joint synthesis of images and segmentation masks with generative adversarial networks (GANs) is promising to reduce the effort needed for collecting image data with pixel-wise annotations. However, to learn high-fidelity image-mask synthesis, existing GAN approaches first need a pre-training phase requiring large amounts of image data, which limits their utilization in restricted image domains. In this work, we take a step to reduce this limitation, introducing the task of one-shot image-mask synthesis. We aim to generate diverse images and their segmentation masks given only a single labelled example, and assuming, contrary to previous models, no access to any pre-training data. To this end, inspired by the recent architectural developments of single-image GANs, we introduce our OSMIS model which enables the synthesis of segmentation masks that are precisely aligned to the generated images in the one-shot regime. Besides achieving the high fidelity of generated masks, OSMIS outperforms state-of-the-art single-image GAN models in image synthesis quality and diversity. In addition, despite not using any additional data, OSMIS demonstrates an impressive ability to serve as a source of useful data augmentation for one-shot segmentation applications, providing performance gains that are complementary to standard data augmentation techniques. Code is available at https://github.com/ boschresearch/one-shot-synthesis