Self-attention-based non-linear basis transformations for compact latent space modelling of dynamic optical fibre transmission matrices

TL;DR

This paper introduces a novel self-attention-based method for transforming dynamic optical fibre matrices into compact, sparse, and invertible representations, improving modelling efficiency for real-world medical imaging applications.

Contribution

It proposes a new neural network approach using self-attention layers to transform fibre matrices into low-dimensional bases, addressing non-linearity and dynamics in optical fibre modelling.

Findings

Significant increase in basis sparsity with participation ratio between 0.01 and 0.11.

Achieved less than 10% reconstruction error, demonstrating effective invertibility.

Demonstrated effectiveness across diverse fibre matrix datasets.

Abstract

Multimode optical fibres are hair-thin strands of glass that efficiently transport light. They promise next-generation medical endoscopes that provide unprecedented sub-cellular image resolution deep inside the body. However, confining light to such fibres means that images are inherently scrambled in transit. Conventionally, this scrambling has been compensated by pre-calibrating how a specific fibre scrambles light and solving a stationary linear matrix equation that represents a physical model of the fibre. However, as the technology develops towards real-world deployment, the unscrambling process must account for dynamic changes in the matrix representing the fibre's effect on light, due to factors such as movement and temperature shifts, and non-linearities resulting from the inaccessibility of the fibre tip when inside the body. Such complex, dynamic and nonlinear behaviour is well-suited to approximation by neural networks, but most leading image reconstruction networks rely on convolutional layers, which assume strong correlations between adjacent pixels, a strong inductive bias that is inappropriate for fibre matrices which may be expressed in a range of arbitrary coordinate representations with long-range correlations. We introduce a new concept that uses self-attention layers to dynamically transform the coordinate representations of varying fibre matrices to a basis that admits compact, low-dimensional representations suitable for further processing. We demonstrate the effectiveness of this approach on diverse fibre matrix datasets. We show our models significantly improve the sparsity of fibre bases in their transformed bases with a participation ratio, p, as a measure of sparsity, of between 0.01 and 0.11. Further, we show that these transformed representations admit reconstruction of the original matrices with < 10% reconstruction error, demonstrating the invertibility.