U-aggregation: Unsupervised Aggregation of Multiple Learning Algorithms

TL;DR

U-aggregation is an unsupervised method that combines multiple pre-trained models to improve performance in new populations without requiring labeled data, addressing heterogeneity and adversarial models.

Contribution

The paper introduces U-aggregation, a novel unsupervised model aggregation technique that handles heteroskedasticity and adversarial models, with theoretical analysis and real-world genetic risk prediction applications.

Findings

U-aggregation improves risk estimation accuracy in simulations.

The method effectively integrates multiple models without labeled data.

Application to genetic risk prediction demonstrates practical utility.

Abstract

Across various domains, the growing advocacy for open science and open-source machine learning has made an increasing number of models publicly available. These models allow practitioners to integrate them into their own contexts, reducing the need for extensive data labeling, training, and calibration. However, selecting the best model for a specific target population remains challenging due to issues like limited transferability, data heterogeneity, and the difficulty of obtaining true labels or outcomes in real-world settings. In this paper, we propose an unsupervised model aggregation method, U-aggregation, designed to integrate multiple pre-trained models for enhanced and robust performance in new populations. Unlike existing supervised model aggregation or super learner approaches, U-aggregation assumes no observed labels or outcomes in the target population. Our method addresses limitations in existing unsupervised model aggregation techniques by accommodating more realistic settings, including heteroskedasticity at both the model and individual levels, and the presence of adversarial models. Drawing on insights from random matrix theory, U-aggregation incorporates a variance stabilization step and an iterative sparse signal recovery process. These steps improve the estimation of individuals' true underlying risks in the target population and evaluate the relative performance of candidate models. We provide a theoretical investigation and systematic numerical experiments to elucidate the properties of U-aggregation. We demonstrate its potential real-world application by using U-aggregation to enhance genetic risk prediction of complex traits, leveraging publicly available models from the PGS Catalog.