Semantic denoising autoencoders for retinal optical coherence tomography

TL;DR

This paper introduces a novel denoising autoencoder for retinal OCT images that preserves disease features and improves image quality and classification accuracy over existing methods.

Contribution

It combines a deep convolutional autoencoder with a pre-trained ResNet classifier as a regularizer, effectively denoising images while maintaining diagnostic details.

Findings

Higher PSNR of 31.2 dB compared to 29.4 dB.

Increased classification accuracy to 85.0% from 70.3%.

Preserves disease details without blurring.

Abstract

Noise in speckle-prone optical coherence tomography tends to obfuscate important details necessary for medical diagnosis. In this paper, a denoising approach that preserves disease characteristics on retinal optical coherence tomography images in ophthalmology is presented. By combining a deep convolutional autoencoder with a priorly trained ResNet image classifier as regularizer, the perceptibility of delicate details is encouraged and only information-less background noise is filtered out. With our approach, higher peak signal-to-noise ratios with $ \mathrm{PSNR} = 31.2\,\mathrm{dB} $ and higher classification accuracy of $\mathrm{ACC} = 85.0\,\%$ can be achieved for denoised images compared to state-of-the-art denoising with $ \mathrm{PSNR} = 29.4\,\mathrm{dB} $ or $\mathrm{ACC} = 70.3\,\%$, depending on the method. It is shown that regularized autoencoders are capable of denoising retinal OCT images without blurring details of diseases.

Tables

Table 1. Table 1: Results of denoising reported for test set with mean peak signal-to-noise ratio PSNR in dB and mean classification accuracy ACC in %. Values for corrupted images 𝒙 ~ ~ 𝒙 \tilde{\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}}} are given for comparison. Bold values denote best results.

	corrupted	TV	wavelet	AD	AE (ours)
PSNR	19.2	29.4	28.0	24.6	31.2
ACC	50.2	49.3	52.6	70.3	85.0

Equations

$$L_{\mathrm{AE}}(\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}},\hat{\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}}},\mathchoice{\mbox{\boldmath$\displaystyle y$}}{\mbox{\boldmath$\textstyle y$}}{\mbox{\boldmath$\scriptstyle y$}}{\mbox{\boldmath$\scriptscriptstyle y$}})=L_{r}(\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}},\hat{\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}}})+\alpha L_{c}(\mathchoice{\mbox{\boldmath$\displaystyle y$}}{\mbox{\boldmath$\textstyle y$}}{\mbox{\boldmath$\scriptstyle y$}}{\mbox{\boldmath$\scriptscriptstyle y$}},C(\mathchoice{\mbox{\boldmath$\displaystyle x$}}{\mbox{\boldmath$\textstyle x$}}{\mbox{\boldmath$\scriptstyle x$}}{\mbox{\boldmath$\scriptscriptstyle x$}}))$$

Full text

1 Purpose

Optical coherence tomography (OCT) is the most common imaging technique for diagnosis in ophthalmology. However, due to image acquisition based on interference of coherent light, OCT suffers from speckle noise. This results in grainy images with low contrast where the diagnosis of medical conditions requires trained expert observers. Denoising of OCT has been addressed in the literature already and can be separated into two categories[1]. The first one employs denoising during OCT acquisition by e.g. averaging multiple frames of the same object. This prolongs the acquisition process and is therefore not applicable for dynamic objects. The second category comprises post-processing methods such as median, bilateral, wavelet-based or other linear and nonlinear filtering techniques. These can be executed in real time but are prone not only to blurring the image, but also to erasing important disease-related details in that process. This paper describes a domain-specific post-processing method for denoising OCT images with machine learning and more specific convolutional autoencoders (AE) while maintaining disease characteristics.

2 Methods

The dataset used in this paper contains 84,484 retinal OCT images from 4,657 patients showing the disease states drusen, diabetic macular edema (DME), choroidal neovascularization (CNV) and normal and is publicly available[2]. First, a ResNet-34 image classifier $C$ pretrained on ImageNet is fine-tuned on the dataset[3]. This acts as medical expert as it has been shown that the performance of convolutional neural networks (CNNs) in classifying retinal conditions is on par to that of trained ophthalmologists [2]. Second, the ErfNet CNN autoencoder is trained to reconstruct input images $\textstyle x$ corrupted by additive gaussian white noise resulting in \mathchoice{\mbox{\boldmath\displaystyle\tilde{x}}}{\mbox{\boldmath\textstyle\tilde{x}}}{\mbox{\boldmath\scriptstyle\tilde{x}}}{\mbox{\boldmath\scriptscriptstyle\tilde{x}}}=\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}+\mathchoice{\mbox{\boldmath\displaystyle c}}{\mbox{\boldmath\textstyle c}}{\mbox{\boldmath\scriptstyle c}}{\mbox{\boldmath\scriptscriptstyle c}} with \mathchoice{\mbox{\boldmath\displaystyle c}}{\mbox{\boldmath\textstyle c}}{\mbox{\boldmath\scriptstyle c}}{\mbox{\boldmath\scriptscriptstyle c}}\sim\mathcal{N}(\mathchoice{\mbox{\boldmath\displaystyle 0}}{\mbox{\boldmath\textstyle 0}}{\mbox{\boldmath\scriptstyle 0}}{\mbox{\boldmath\scriptscriptstyle 0}},0.1\,\mathbf{I})[4]. In general, an AE consists of two components. The encoder $E$ takes an input image $\textstyle x$, or in our case $\textstyle\tilde{x}$, and maps it from high dimension into low-dimensional, latent representation $\textstyle z$. This is then fed into the decoder $D$ and mapped back to a reconstructed image \hat{\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}} in input space. The parameters of the AE are optimized by minimizing the pixel-wise mean squared reconstruction error L_{r}(\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}},\hat{\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}}). Essentially, an autoencoder learns a low-dimensional representation similar to principal component analysis (PCA). When training with a large dataset, noise tends to “average out” and the AE reconstructs distinct and relevant (noise-free) image features. In order to promote enhancement of these features, the trained ResNet with fixed weights is used as additional optimization criterion $L_{c}$. It is applied to the reconstructed, denoised image and tries to predict the retinal disease class. This regularizes the AE during training and enhances disease characteristics in denoised images. The proposed approach is therefore optimized using the weighted loss function

[TABLE]

with denoised corrupted image \hat{\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}}=D(E(\tilde{\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}})), true disease label $\textstyle y$ of image $\textstyle x$ and cross entropy for $L_{c}$. Weighting factor for $L_{c}$ was empirically set to $\alpha=0.1$. The aforementioned method is implemented with PyTorch 1.0 and trained for 200 epochs using the Adam optimizer with an initial learning rate of $\eta=10^{-4}$[5]. A reduce-on-plateau learning rate scheduling is realized to reduce $\eta$ with a factor of $10^{-1}$ when observing saturation of the validation loss. The weight configuration with lowest loss value on the validation set is chosen for testing (early stopping).

3 Results

The CNN are optimized using 79,484 OCT images for training, 4,000 for validation and 1,000 for testing. To assess denoising performance, the proposed method is compared to total variation (TV) minimization[6], wavelet[7], and anisotropic diffusion (AD)[8] denoising regarding peak signal-to-noise ratio (PSNR) and classification performance of ResNet.

The results are summarized in Tab. 1. Our approach not only provides the highest disease classification accuracy with $\mathrm{ACC}=85.0\,\%$ after denoising, but also has the highest peak signal-to-noise ration with $\mathrm{PSNR}=31.2\,\mathrm{dB}$ compared to the other methods.

Fig. 1 visualizes qualitative results for example OCT from the test set showing different disease conditions. The methods are used to restore the input image $\textstyle x$ (first column) from the corrupted image \tilde{\mathchoice{\mbox{\boldmath\displaystyle x}}{\mbox{\boldmath\textstyle x}}{\mbox{\boldmath\scriptstyle x}}{\mbox{\boldmath\scriptscriptstyle x}}} (second column). In contrast to state-of-the-art denoising, our approach is able to distinctively preserve the retinal layers while removing speckle noise. Pathological alterations of the retina are clearly visible and the explanatory power for diagnosis is not reduced. Mean processing time of AE for one image is 13.1 ms on an NVIDIA GeForce GTX 1080 Ti.

4 Conclusion

It has been shown that convolutional AEs are capable of denoising retinal OCT images without suppressing characteristics of diseases. This was achieved by regularizing the denoising AE during training with another CNN, which was previously trained for disease classification. The trained decoder can also be used to generate new images by sampling the latent space. Future work therefore aims on variational AEs and generative adversarial networks for OCT denoising. It should be noted, however, that speckle noise can also contain significant information as it creates a unique fingerprint of tissue. This information cannot be interpreted by humans, and CNNs can be valuable tools to acquire and utilize this information in the future.

Conflict of Interest

The authors declare that they have no conflict of interest.

Formal Consent

The medical images used in this article were made available to the public in a previous study [2], therefore formal consent is not required.