FurryGAN: High Quality Foreground-aware Image Synthesis

TL;DR

FurryGAN is a novel generative adversarial network that produces high-quality, detailed foreground masks and images in an unsupervised manner by combining realism constraints, multi-scale masks, and auxiliary mask prediction.

Contribution

It introduces a new framework with three key components that effectively generate realistic foreground-aware images and masks without supervision.

Findings

Produces highly detailed alpha masks covering complex features like hair and fur.

Achieves realistic image synthesis with meaningful foreground-background separation.

Operates fully unsupervised, eliminating the need for labeled data.

Abstract

Foreground-aware image synthesis aims to generate images as well as their foreground masks. A common approach is to formulate an image as an masked blending of a foreground image and a background image. It is a challenging problem because it is prone to reach the trivial solution where either image overwhelms the other, i.e., the masks become completely full or empty, and the foreground and background are not meaningfully separated. We present FurryGAN with three key components: 1) imposing both the foreground image and the composite image to be realistic, 2) designing a mask as a combination of coarse and fine masks, and 3) guiding the generator by an auxiliary mask predictor in the discriminator. Our method produces realistic images with remarkably detailed alpha masks which cover hair, fur, and whiskers in a fully unsupervised manner.