Learning to reconstruct signals with inexact sensing operator via knowledge distillation

TL;DR

This paper introduces a knowledge distillation approach for signal reconstruction when the sensing operator is inexact, improving performance in optical imaging and wireless communication systems.

Contribution

It proposes a novel framework where a teacher model trained with accurate sensing guides a student model with inexact sensing, enhancing reconstruction accuracy.

Findings

Improved signal recovery in optical imaging with miscalibrated systems

Enhanced MIMO detection with inexact channel matrices

Demonstrated effectiveness of knowledge distillation in practical scenarios

Abstract

In computational optical imaging and wireless communications, signals are acquired through linear coded and noisy projections, which are recovered through computational algorithms. Deep model-based approaches, i.e., neural networks incorporating the sensing operators, are the state-of-the-art for signal recovery. However, these methods require exact knowledge of the sensing operator, which is often unavailable in practice, leading to performance degradation. Consequently, we propose a new recovery paradigm based on knowledge distillation. A teacher model, trained with full or almost exact knowledge of a synthetic sensing operator, guides a student model with an inexact real sensing operator. The teacher is interpreted as a relaxation of the student since it solves a problem with fewer constraints, which can guide the student to achieve higher performance. We demonstrate the improvement of signal reconstruction in computational optical imaging for single-pixel imaging with miscalibrated coded apertures systems and multiple-input multiple-output symbols detection with inexact channel matrix.