Detection of weak signals under arbitrary noise distributions

TL;DR

This paper introduces a hybrid neural network and Rao detector framework that improves weak signal detection in complex, non-Gaussian noise environments, achieving near-optimal performance and robustness across various applications.

Contribution

It presents a novel hybrid approach combining neural networks with Rao detection to handle arbitrary noise distributions without relying on simplified models.

Findings

Significantly outperforms traditional methods in simulated data

Achieves robust detection in real-world magnetic sensor data

Maintains asymptotic optimality under challenging noise conditions

Abstract

Detecting weak signals buried in complex, non-Gaussian noise is a fundamental challenge in science and engineering, with applications ranging from radar systems and communications to industrial monitoring and gravitational wave detection. The Rao detector, a key concept in this domain, achieves asymptotically optimal performance as the number of measurements increases, but requires precise knowledge of the data's statistical properties, often relying on simplified noise models. We propose a hybrid framework that combines a lightweight neural network with the Rao detection framework to address this limitation. The neural network, trained on noise-only data, learns the optimal multivariate nonlinearity, transforming noisy data to enhance signal detectability. The newly introduced LRao detector then fully extracts the signal information, achieving asymptotically optimal performance even under challenging noise conditions. Validated on both simulated and real-world magnetic sensor data, our method significantly outperforms conventional approaches. By bridging data-driven techniques with model-based signal processing, it offers a robust and interpretable solution for signal detection across diverse applications.