Analysis of Macula on Color Fundus Images Using Heightmap Reconstruction Through Deep Learning

TL;DR

This paper introduces a deep learning-based method using cGANs and deep supervision to accurately reconstruct 3D heightmaps of the macula from 2D fundus images, aiding diagnosis of macular disorders.

Contribution

It presents a novel generator architecture that improves heightmap reconstruction by progressive refinement and deep supervision, outperforming existing methods.

Findings

Outperforms state-of-the-art methods in heightmap reconstruction

Provides more detailed and accurate macula heightmaps

Enhances diagnostic information for ophthalmologists

Abstract

For medical diagnosis based on retinal images, a clear understanding of 3D structure is often required but due to the 2D nature of images captured, we cannot infer that information. However, by utilizing 3D reconstruction methods, we can recover the height information of the macula area on a fundus image which can be helpful for diagnosis and screening of macular disorders. Recent approaches have used shading information for heightmap prediction but their output was not accurate since they ignored the dependency between nearby pixels and only utilized shading information. Additionally, other methods were dependent on the availability of more than one image of the retina which is not available in practice. In this paper, motivated by the success of Conditional Generative Adversarial Networks(cGANs) and deeply supervised networks, we propose a novel architecture for the generator which enhances the details and the quality of output by progressive refinement and the use of deep supervision to reconstruct the height information of macula on a color fundus image. Comparisons on our own dataset illustrate that the proposed method outperforms all of the state-of-the-art methods in image translation and medical image translation on this particular task. Additionally, perceptual studies also indicate that the proposed method can provide additional information for ophthalmologists for diagnosis.