Learning U-Statistics with Active Inference

TL;DR

This paper introduces an active inference framework for U-statistics that improves estimation efficiency by selectively querying labels, applicable to statistical inference and empirical risk minimization.

Contribution

It develops a novel active inference approach for U-statistics, including optimal sampling strategies and extensions to empirical risk minimization, with demonstrated empirical benefits.

Findings

Significant efficiency gains over baseline methods in real datasets

Optimal sampling rule minimizes variance of the augmented inverse probability weighting U-statistic

Framework maintains valid statistical inference and coverage

Abstract

$U$-statistics play a central role in statistical inference. In many modern applications, however, acquiring the labels required for $U$-statistics is costly. Motivated by recent advances in active inference, we develop an active inference framework for $U$-statistics that selectively queries informative labels to improve estimation efficiency under a fixed labeling budget, while preserving valid statistical inference. Our approach is built on the augmented inverse probability weighting $U$-statistic, which is designed to incorporate the sampling rule and machine learning predictions. We characterize the optimal sampling rule that minimizes its variance and design practical sampling strategies. We further extend the framework to $U$-statistic-based empirical risk minimization. Experiments on real datasets demonstrate substantial gains in estimation efficiency over baseline methods, while maintaining target coverage.