Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencoder

TL;DR

This paper introduces LSSAE, a probabilistic autoencoder model that captures the continuous evolution of domain shifts in non-stationary environments, improving out-of-distribution generalization.

Contribution

It formulates evolving domain generalization as a problem and proposes LSSAE to model continuous latent structure changes, addressing covariate and concept shifts.

Findings

LSSAE outperforms existing methods on synthetic datasets.

LSSAE achieves superior results on real-world evolving domain tasks.

The model effectively captures continuous domain evolution.

Abstract

Domain generalization aims to improve the generalization capability of machine learning systems to out-of-distribution (OOD) data. Existing domain generalization techniques embark upon stationary and discrete environments to tackle the generalization issue caused by OOD data. However, many real-world tasks in non-stationary environments (e.g. self-driven car system, sensor measures) involve more complex and continuously evolving domain drift, which raises new challenges for the problem of domain generalization. In this paper, we formulate the aforementioned setting as the problem of evolving domain generalization. Specifically, we propose to introduce a probabilistic framework called Latent Structure-aware Sequential Autoencoder (LSSAE) to tackle the problem of evolving domain generalization via exploring the underlying continuous structure in the latent space of deep neural networks, where we aim to identify two major factors namely covariate shift and concept shift accounting for distribution shift in non-stationary environments. Experimental results on both synthetic and real-world datasets show that LSSAE can lead to superior performances based on the evolving domain generalization setting.