Domain Generalization with Small Data

TL;DR

This paper introduces a probabilistic domain generalization method that learns domain-invariant representations using a novel probabilistic MMD and contrastive loss, improving performance on medical datasets with limited samples.

Contribution

It proposes a new probabilistic framework for domain generalization that extends MMD and contrastive loss to probabilistic embeddings, addressing small data challenges.

Findings

Outperforms state-of-the-art methods on three medical datasets.

Effectively captures distributional information with probabilistic embeddings.

Enhances domain invariance in low-data scenarios.

Abstract

In this work, we propose to tackle the problem of domain generalization in the context of \textit{insufficient samples}. Instead of extracting latent feature embeddings based on deterministic models, we propose to learn a domain-invariant representation based on the probabilistic framework by mapping each data point into probabilistic embeddings. Specifically, we first extend empirical maximum mean discrepancy (MMD) to a novel probabilistic MMD that can measure the discrepancy between mixture distributions (i.e., source domains) consisting of a series of latent distributions rather than latent points. Moreover, instead of imposing the contrastive semantic alignment (CSA) loss based on pairs of latent points, a novel probabilistic CSA loss encourages positive probabilistic embedding pairs to be closer while pulling other negative ones apart. Benefiting from the learned representation captured by probabilistic models, our proposed method can marriage the measurement on the \textit{distribution over distributions} (i.e., the global perspective alignment) and the distribution-based contrastive semantic alignment (i.e., the local perspective alignment). Extensive experimental results on three challenging medical datasets show the effectiveness of our proposed method in the context of insufficient data compared with state-of-the-art methods.