Content-Consistent Generation of Realistic Eyes with Style

TL;DR

This paper presents methods for generating realistic, person-specific eye images that preserve semantic content and style from limited reference images, improving data augmentation for eye-related tasks.

Contribution

It introduces two approaches for content-consistent eye image synthesis, including a winning method from a challenge and a new multi-scale style-content injection technique.

Findings

Won the OpenEDS Synthetic Eye Generation Challenge at ICCV 2019

Proposed a novel multi-scale style and content injection method

Demonstrated high-quality, person-specific eye image generation

Abstract

Accurately labeled real-world training data can be scarce, and hence recent works adapt, modify or generate images to boost target datasets. However, retaining relevant details from input data in the generated images is challenging and failure could be critical to the performance on the final task. In this work, we synthesize person-specific eye images that satisfy a given semantic segmentation mask (content), while following the style of a specified person from only a few reference images. We introduce two approaches, (a) one used to win the OpenEDS Synthetic Eye Generation Challenge at ICCV 2019, and (b) a principled approach to solving the problem involving simultaneous injection of style and content information at multiple scales. Our implementation is available at https://github.com/mcbuehler/Seg2Eye.