Roles of scattered and ballistic photons in imaging through scattering media: a deep learning-based study

TL;DR

This study investigates how deep learning can differentiate and utilize scattered and ballistic photons for imaging through complex media, revealing distinct physical mechanisms and the importance of ballistic photons for neural network generalization.

Contribution

The paper uncovers the different roles of scattered and ballistic photons in deep learning-based imaging and links the generalization ability of neural networks to ballistic photon contributions.

Findings

Deep learning can extract images from both scattered and ballistic light.

Scattering acts as an encryption key, while ballistic light provides stable information.

Ballistic photon contribution underpins neural network generalization across diffusers.

Abstract

Scattering of light in complex media scrambles optical wavefronts and breaks the principles of conventional imaging methods. For decades, researchers have endeavored to conquer the problem by inventing approaches such as adaptive optics, iterative wavefront shaping, and transmission matrix measurement. That said, imaging through/into thick scattering media remains challenging to date. With the rapid development of computing power, deep learning has been introduced and shown potentials to reconstruct target information through complex media or from rough surfaces. But it also fails once coming to optically thick media where ballistic photons become negligible. Here, instead of treating deep learning only as an image extraction method, whose best-selling advantage is to avoid complicate physical models, we exploit it as a tool to explore the underlying physical principles. By adjusting the weights of ballistic and scattered photons through a random phasemask, it is found that although deep learning can extract images from both scattered and ballistic light, the mechanisms are different: scattering may function as an encryption key and decryption from scattered light is key sensitive, while extraction from ballistic light is stable. Based on this finding, it is hypothesized and experimentally confirmed that the foundation of the generalization capability of trained neural networks for different diffusers can trace back to the contribution of ballistic photons, even though their weights of photon counting in detection are not that significant. Moreover, the study may pave an avenue for using deep learning as a probe in exploring the unknown physical principles in various fields.