# Real‐Time High‐Definition Hyperspectral Endoscopy via Spatial‐Temporal Low‐Frequency‐Stochastic Spectral Encoding

## TL;DR

This paper introduces a new hyperspectral endoscopy method that captures high-definition images in real-time, allowing detailed visualization of tissue features and changes during medical procedures.

## Contribution

The novel spatial-temporal spectral encoding approach with low-frequency stochastic filters and an attention network enables real-time hyperspectral imaging.

## Key findings

- The prototype system achieves 20 Hz frame rate with high-definition resolution and 67 spectral channels.
- The method captures fine superficial tissue features, including capillaries as small as 37 µm in diameter.
- It enables visualization of compositional changes during photodynamic therapy and hepatic ischemia.

## Abstract

Hyperspectral endoscopy enables minimally invasive visualization of both structural and compositional information, offering promising potential for the accurate assessment of in vivo physiological and pathological conditions. However, current hyperspectral endoscopy suffers from low frame rate, hindering the clear capture of in vivo tissues in motion, restricting in vivo diagnostics, efficacy assessment, or risk monitoring during minimally invasive procedures. Here we propose a hyperspectral endoscopy by developing a spatial‐temporal spectral encoding approach based on low‐frequency stochastic filters combined with an encoding‐guided spectral attention network (ESANet) to reconstruct the hyperspectral image with low latency. A prototype system is developed to achieve real‐time frame rate (20 Hz), high‐definition resolution (full pixels), 67 spectral channels spanning 420–750 nm. It can overcome the continuous motion of in vivo tissue to provide hyperspectral images with fine superficial features, including capillary as small as around 37 µm in diameter, reveal the distinct spectra characteristics for diverse types of organs, and enable visualization of rapid and subtle compositional changes in two representative processes: photodynamic therapy and hepatic ischemia. With minimal hardware modifications, the proposed scheme provides a cost‐efficient and easily adaptable solution for hyperspectral endoscopy as well as broader application scenarios.

A real‐time, high‐definition hyperspectral endoscopy is enabled by developing a spatial‐temporal spectral encoding approach based on low‐frequency stochastic filters combined with an encoding‐guided attention network. It provides hyperspectral image of in vivo tissue with fine superficial features, enables visualization of rapid and subtle compositional changes, and offers potential for accurate diagnosis and precise treatment guidance.