Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

TL;DR

This paper introduces a deep learning framework that disentangles intrinsic signals from measurement artifacts in multi-sensor data, enabling more accurate scientific analysis and instrument-independent comparisons, demonstrated on astrophysical galaxy images.

Contribution

The proposed method uses overlapping observations, dual-encoder architecture, and counterfactual objectives to explicitly separate physical signals from sensor-specific distortions.

Findings

Effective disentanglement of signals and artifacts in astrophysical data.

Enables counterfactual view generation and unconfounded parameter inference.

Applicable as a general self-supervised pretraining approach for multi-modal data.

Abstract

Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.