Multi-dimensional domain generalization with low-rank structures

TL;DR

This paper introduces a tensor-based approach for multi-dimensional domain generalization in linear regression, effectively handling underrepresented sub-populations with limited data, and provides theoretical guarantees and empirical validation.

Contribution

It proposes a novel tensor completion method leveraging group label structures for robust domain generalization in linear models, with proven optimality and practical validation.

Findings

The method achieves minimax optimality.

It improves prediction accuracy for minority groups.

The approach is validated on real-world data.

Abstract

In conventional statistical and machine learning methods, it is typically assumed that the test data are identically distributed with the training data. However, this assumption does not always hold, especially in applications where the target population are not well-represented in the training data. This is a notable issue in health-related studies, where specific ethnic populations may be underrepresented, posing a significant challenge for researchers aiming to make statistical inferences about these minority groups. In this work, we present a novel approach to addressing this challenge in linear regression models. We organize the model parameters for all the sub-populations into a tensor. By studying a structured tensor completion problem, we can achieve robust domain generalization, i.e., learning about sub-populations with limited or no available data. Our method novelly leverages the structure of group labels and it can produce more reliable and interpretable generalization results. We establish rigorous theoretical guarantees for the proposed method and demonstrate its minimax optimality. To validate the effectiveness of our approach, we conduct extensive numerical experiments and a real data study focused on education level prediction for multiple ethnic groups, comparing our results with those obtained using other existing methods.