Radiative-Structured Neural Operator for Continuous Spectral Super-Resolution

TL;DR

This paper introduces RSNO, a physics-informed neural operator that learns continuous spectral mappings for hyperspectral image reconstruction, improving realism and applicability in spectral super-resolution tasks.

Contribution

The paper proposes RSNO, a novel neural operator framework that enforces physical radiative constraints and learns continuous spectral mappings for hyperspectral super-resolution.

Findings

RSNO outperforms existing methods in spectral super-resolution accuracy.

The angular-consistent projection effectively enforces physical constraints.

Theoretical analysis confirms the optimality of the projection method.

Abstract

Spectral super-resolution (SSR) aims to reconstruct hyperspectral images (HSIs) from multispectral observations, with broad applications in computer vision and remote sensing. Deep learning-based methods have been widely used, but they often treat spectra as discrete vectors learned from data, rather than continuous curves constrained by physics principles, leading to unrealistic predictions and limited applicability. To address this challenge, we propose the Radiative-Structured Neural Operator (RSNO), which learns a continuous mapping for spectral super-resolution while enforcing physical consistency under the radiative prior. The proposed RSNO consists of three stages: upsampling, reconstruction, and refinement. In the upsampling stage, we leverage prior information to expand the input multispectral image, producing a physically plausible hyperspectral estimate. Subsequently, we adopt a neural operator backbone in the reconstruction stage to learn a continuous mapping across the spectral domain. Finally, the refinement stage imposes a hard constraint on the output HSI to eliminate color distortion. The upsampling and refinement stages are implemented via the proposed angular-consistent projection (ACP), which is derived from a non-convex optimization problem. Moreover, we theoretically demonstrated the optimality of ACP by null-space decomposition. Various experiments validate the effectiveness of the proposed approach in both discrete and continuous spectral super-resolution.