Multivariate Conformal Selection

TL;DR

This paper introduces Multivariate Conformal Selection (mCS), a novel framework extending conformal selection to multivariate responses, enabling effective candidate selection with FDR control in complex datasets.

Contribution

We propose mCS, a new multivariate conformal selection method with regional monotonicity and two variants, improving selection power while controlling FDR.

Findings

mCS outperforms existing methods in simulated datasets.

mCS maintains FDR control in real-world applications.

mCS variants adapt to different scoring strategies.

Abstract

Selecting high-quality candidates from large datasets is critical in applications such as drug discovery, precision medicine, and alignment of large language models (LLMs). While Conformal Selection (CS) provides rigorous uncertainty quantification, it is limited to univariate responses and scalar criteria. To address this issue, we propose Multivariate Conformal Selection (mCS), a generalization of CS designed for multivariate response settings. Our method introduces regional monotonicity and employs multivariate nonconformity scores to construct conformal p-values, enabling finite-sample False Discovery Rate (FDR) control. We present two variants: mCS-dist, using distance-based scores, and mCS-learn, which learns optimal scores via differentiable optimization. Experiments on simulated and real-world datasets demonstrate that mCS significantly improves selection power while maintaining FDR control, establishing it as a robust framework for multivariate selection tasks.