Learning Robust Spectral Dynamics for Temporal Domain Generalization

TL;DR

This paper introduces FreKoo, a spectral analysis-based method for temporal domain generalization that effectively models complex distribution shifts using frequency decomposition and Koopman operator extrapolation, with theoretical guarantees and superior empirical performance.

Contribution

The paper proposes a novel spectral approach using Fourier analysis and Koopman operators for robust temporal domain generalization, addressing complex real-world drifts.

Findings

FreKoo outperforms state-of-the-art methods in real-world streaming scenarios.

Spectral decomposition effectively captures diverse drift patterns.

Theoretical analysis provides stability and generalization guarantees.

Abstract

Modern machine learning models struggle to maintain performance in dynamic environments where temporal distribution shifts, \emph{i.e., concept drift}, are prevalent. Temporal Domain Generalization (TDG) seeks to enable model generalization across evolving domains, yet existing approaches typically assume smooth incremental changes, struggling with complex real-world drifts involving long-term structure (incremental evolution/periodicity) and local uncertainties. To overcome these limitations, we introduce FreKoo, which tackles these challenges via a novel frequency-domain analysis of parameter trajectories. It leverages the Fourier transform to disentangle parameter evolution into distinct spectral bands. Specifically, low-frequency component with dominant dynamics are learned and extrapolated using the Koopman operator, robustly capturing diverse drift patterns including both incremental and periodicity. Simultaneously, potentially disruptive high-frequency variations are smoothed via targeted temporal regularization, preventing overfitting to transient noise and domain uncertainties. In addition, this dual spectral strategy is rigorously grounded through theoretical analysis, providing stability guarantees for the Koopman prediction, a principled Bayesian justification for the high-frequency regularization, and culminating in a multiscale generalization bound connecting spectral dynamics to improved generalization. Extensive experiments demonstrate FreKoo's significant superiority over SOTA TDG approaches, particularly excelling in real-world streaming scenarios with complex drifts and uncertainties.