Selection from Hierarchical Data with Conformal e-values

TL;DR

This paper develops conformal e-value methods for reliable selection in hierarchical data, enabling false discovery rate control and demonstrating effectiveness in real-world applications.

Contribution

It introduces and compares two new conformal e-value approaches for hierarchical data, extending distribution-free inference beyond i.i.d. settings.

Findings

Both methods achieve valid FDR control.

Subsampling method offers higher power, hierarchical method is more stable.

Empirical results confirm practical effectiveness.

Abstract

Distribution-free predictive inference beyond the construction of prediction sets has gained a lot of interest in recent applications. One such application is the selection task, where the objective is to design a reliable selection rule to pick out individuals with desired unobserved outcomes while controlling the error rate. In this work, we address the selection problem in the context of hierarchical data, where groups of observations may exhibit distinct within-group distributions. This generalizes existing techniques beyond the standard i.i.d./exchangeable data settings. As a correction, For hierarchical data, we introduce methods to construct valid conformal e-values, enabling control of the false discovery rate (FDR) through the e-BH procedure. In particular, we introduce and compare two approaches -- subsampling conformal e-values and hierarchical conformal e-values. Empirical results demonstrate that both approaches achieve valid FDR control while highlighting a tradeoff between stability and power. The subsampling-based method, though random, typically offers higher power, whereas the hierarchical approach, being deterministic, tends to be slightly less powerful. The effectiveness of the proposed methods is illustrated in two real-world applications.