Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction

TL;DR

This paper introduces a novel training approach for deep neural networks to enhance single-image super-resolution in endomicroscopy, leveraging synthetic data and video-registration techniques to improve image quality.

Contribution

It proposes a synthetic data generation method for training exemplar-based DNNs, improving super-resolution of pCLE images without relying on computationally demanding frame alignment.

Findings

Enhanced image quality demonstrated through extensive IQA metrics

Effective super-resolution achieved with the proposed training strategy

Improved performance of multiple state-of-the-art DNNs on pCLE data

Abstract

Purpose: Probe-based Confocal Laser Endomicroscopy (pCLE) is a recent imaging modality that allows performing in vivo optical biopsies. The design of pCLE hardware, and its reliance on an optical fibre bundle, fundamentally limits the image quality with a few tens of thousands fibres, each acting as the equivalent of a single-pixel detector, assembled into a single fibre bundle. Video-registration techniques can be used to estimate high-resolution (HR) images by exploiting the temporal information contained in a sequence of low-resolution (LR) images. However, the alignment of LR frames, required for the fusion, is computationally demanding and prone to artefacts. Methods: In this work, we propose a novel synthetic data generation approach to train exemplar-based Deep Neural Networks (DNNs). HR pCLE images with enhanced quality are recovered by the models trained on pairs of estimated HR images (generated by the video-registration algorithm) and realistic synthetic LR images. Performance of three different state-of-the-art DNNs techniques were analysed on a Smart Atlas database of 8806 images from 238 pCLE video sequences. The results were validated through an extensive Image Quality Assessment (IQA) that takes into account different quality scores, including a Mean Opinion Score (MOS). Results: Results indicate that the proposed solution produces an effective improvement in the quality of the obtained reconstructed image. Conclusion: The proposed training strategy and associated DNNs allows us to perform convincing super-resolution of pCLE images.