Photoacoustic image synthesis with generative adversarial networks

TL;DR

This paper introduces a novel GAN-based method for synthesizing realistic photoacoustic images by separately generating tissue morphology and optical properties, addressing domain gap issues in training data for improved deep learning applications.

Contribution

It proposes a two-step GAN approach for realistic photoacoustic image synthesis, enhancing the quality of simulated data for quantitative PAT.

Findings

GAN-generated images are more realistic than traditional methods

Improved synthetic data benefits deep learning-based PAT

Potential to advance quantitative photoacoustic tomography

Abstract

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).