Noise-robust Contrastive Learning for Critical Transition Detection in Dynamical Systems

TL;DR

This paper introduces a noise-robust contrastive learning method using a singular value decomposition-based neural network and semi-orthogonality constraints, improving critical transition detection in noisy dynamical systems.

Contribution

The paper presents a novel neural network architecture with semi-orthogonality constraints that enhances noise resistance and efficiency in contrastive learning for critical transition detection.

Findings

Matches traditional contrastive learning in accuracy

More lightweight and computationally efficient

Significantly more resistant to noise

Abstract

Detecting critical transitions in complex, noisy time-series data is a fundamental challenge across science and engineering. Such transitions may be anticipated by the emergence of a low-dimensional order parameter, whose signature is often masked by high-amplitude stochastic variability. Standard contrastive learning approaches based on deep neural networks, while promising for detecting critical transitions, are often overparameterized and sensitive to irrelevant noise, leading to inaccurate identification of critical points. To address these limitations, we propose a neural network architecture, constructed using singular value decomposition technique, together with a strictly semi-orthogonality-constrained training algorithm, to enhance the performance of traditional contrastive learning. Extensive experiments demonstrate that the proposed method matches the performance of traditional contrastive learning techniques in identifying critical transitions, yet is considerably more lightweight and markedly more resistant to noise.