BiOcularGAN: Bimodal Synthesis and Annotation of Ocular Images

TL;DR

BiOcularGAN is a novel framework that generates large-scale, photorealistic bimodal ocular images with annotations, reducing the need for manual labeling and addressing privacy concerns in ocular image segmentation.

Contribution

It introduces a dual-branch StyleGAN2 model and a semantic mask generator for bimodal image synthesis and annotation, enabling high-quality dataset creation with minimal manual effort.

Findings

Produces high-quality bimodal ocular images and annotations

Improves segmentation model performance across multiple datasets

Reduces manual annotation effort and privacy concerns

Abstract

Current state-of-the-art segmentation techniques for ocular images are critically dependent on large-scale annotated datasets, which are labor-intensive to gather and often raise privacy concerns. In this paper, we present a novel framework, called BiOcularGAN, capable of generating synthetic large-scale datasets of photorealistic (visible light and near-infrared) ocular images, together with corresponding segmentation labels to address these issues. At its core, the framework relies on a novel Dual-Branch StyleGAN2 (DB-StyleGAN2) model that facilitates bimodal image generation, and a Semantic Mask Generator (SMG) component that produces semantic annotations by exploiting latent features of the DB-StyleGAN2 model. We evaluate BiOcularGAN through extensive experiments across five diverse ocular datasets and analyze the effects of bimodal data generation on image quality and the produced annotations. Our experimental results show that BiOcularGAN is able to produce high-quality matching bimodal images and annotations (with minimal manual intervention) that can be used to train highly competitive (deep) segmentation models (in a privacy aware-manner) that perform well across multiple real-world datasets. The source code for the BiOcularGAN framework is publicly available at https://github.com/dariant/BiOcularGAN.