Pre-training Enables Extraordinary All-optical Image Denoising

TL;DR

This paper introduces a pre-training approach for optical neural networks that significantly improves all-optical image denoising, enabling versatile and high-quality noise reduction across various image types.

Contribution

The authors develop a two-step pre-training and fine-tuning method for optical neural networks, achieving superior denoising performance compared to traditional methods.

Findings

Pre-trained optical networks improve PSNR from below 8 dB to above 18 dB.

The same pre-trained model adapts to diverse image styles including handwritten digits, X-rays, and faces.

Optical denoisers enhance vision tasks like face detection and UAV localization in noisy environments.

Abstract

Optical neural networks are emerging as powerful machine learning and information processing tools because of their potential advantages in speed and energy efficiency. The training methods of these physical models, however, remain underexplored compared to their digital counterparts and are leading to suboptimal performance. This paper reports a pre-training-driven approach that leads to snapshot image denoising with substantially improved quality. We demonstrated effective free-space optical denoising by a diffractive network optimized by a two-step process including (1) pre-training using a massive dataset of 3.45 million diverse but simple images and (2) fine-tuning with the corresponding task-specific datasets. Compared to conventional Fourier-domain filtering and directly trained diffractive networks, such a transfer learning process exhibited prominent advantages for denoising images degraded by severe noise, peak signal-to-noise ratio (PSNR) below 8 dB, while preserving fine image features and improving the PSNR to above 18 dB. Importantly, the same pre-trained optical network could be consistently fine-tuned to process degraded images from highly diverse styles ranging from handwritten digits (MNIST) and chest X-rays (ChestMNIST) to CIFAR-10 images and human faces (CelebA). We further demonstrated the critical role of our optical denoisers in vision-based applications, including face detection, plate recognition, and localization of UAVs in noisy conditions.