Sim2Real in Reconstructive Spectroscopy: Deep Learning with Augmented Device-Informed Data Simulation

TL;DR

This paper introduces Sim2Real, a deep learning framework for reconstructing spectral signals in spectroscopy using device-informed simulated data, employing hierarchical data augmentation to bridge the domain gap and achieve fast, accurate results.

Contribution

The work presents a novel hierarchical data augmentation strategy and a neural network architecture tailored for spectral reconstruction from simulated data, addressing domain shift challenges.

Findings

Achieves significant inference speed-up compared to traditional methods.

Attains comparable spectral reconstruction accuracy to state-of-the-art optimization techniques.

Effectively mitigates domain shift using hierarchical data augmentation.

Abstract

This work proposes a deep learning (DL)-based framework, namely Sim2Real, for spectral signal reconstruction in reconstructive spectroscopy, focusing on efficient data sampling and fast inference time. The work focuses on the challenge of reconstructing real-world spectral signals under the extreme setting where only device-informed simulated data are available for training. Such device-informed simulated data are much easier to collect than real-world data but exhibit large distribution shifts from their real-world counterparts. To leverage such simulated data effectively, a hierarchical data augmentation strategy is introduced to mitigate the adverse effects of this domain shift, and a corresponding neural network for the spectral signal reconstruction with our augmented data is designed. Experiments using a real dataset measured from our spectrometer device demonstrate that Sim2Real achieves significant speed-up during the inference while attaining on-par performance with the state-of-the-art optimization-based methods.