Neyman-Pearson Classification under Both Null and Alternative Distributions Shift
Mohammadreza M. Kalan, Yuyang Deng, Eitan J. Neugut, Samory Kpotufe
TL;DR
This paper develops an adaptive transfer learning method for Neyman-Pearson classification that effectively manages shifts in both null and alternative distributions, ensuring error control and avoiding negative transfer.
Contribution
It introduces an adaptive procedure that guarantees improved errors and automatically adapts to the informativeness of the source distribution in Neyman-Pearson classification.
Findings
Guarantees improved Type-I and Type-II errors with informative sources.
Automatically adapts to uninformative sources to prevent negative transfer.
Provides computational guarantees for efficiency.
Abstract
We consider the problem of transfer learning in Neyman-Pearson classification, where the objective is to minimize the error w.r.t. a distribution , subject to the constraint that the error w.r.t. a distribution remains below a prescribed threshold. While transfer learning has been extensively studied in traditional classification, transfer learning in imbalanced classification such as Neyman-Pearson classification has received much less attention. This setting poses unique challenges, as both types of errors must be simultaneously controlled. Existing works address only the case of distribution shift in , whereas in many practical scenarios shifts may occur in both and . We derive an adaptive procedure that not only guarantees improved Type-I and Type-II errors when the source is informative, but also automatically adapt to situations where the…
Peer Reviews
Decision·ICLR 2026 Poster
I was not previously familiar with the NP classification setting, but I find both the problem and the authors’ extensions interesting. The authors also provide sufficient theoretical guarantees for their proposed algorithm, along with supportive simulation studies. In terms of novelty and substance, I believe the paper is worthy of publication at ICLR, although this is not my primary area of expertise.
1. It is somewhat difficult to discern a strong novelty in the transfer-learning extension, even though the authors present solid theoretical results and simulations.
This paper addresses Neyman-Pearson classification within the transfer learning framework, which is highly relevant to the conference. The Neyman-Pearson classification setup is practical due to its importance in high-stakes real-world applications, and transfer learning is motivated by the scarcity of labeled samples in the target domain. The authors present an algorithm with error bound guarantees that avoids negative transfer, even when both the negative and positive source and target distri
The practical scenarios in which $R_{\varphi,\mu_1,T}(h^\ast_{S,T,\alpha}) - R_{\varphi,\mu_1,T}(h^\ast_{T,\alpha})$ is small are unclear. If this term is not small, using only target samples dominates the rate. In such cases, the benefit of transfer learning is not adequately demonstrated. Since $h^\ast_{S,T,\alpha}$ is defined as the maximizer of $R_{\varphi,\mu_1,S}$, the excess error between $h^\ast_{S,T,\alpha}$ and $h^\ast_{T,\alpha}$ could remain large even when the source and target dist
great theoretical results
**Weaknesses.** The paper does not adequately discuss its limitations. In particular, the theoretical generalization analysis appears to rely on boundedness assumptions on the loss or surrogate loss, which is a common but restrictive condition. Moreover, there are alternative transfer learning settings—such as fine-tuning and related approaches discussed in [1] and [2]—that are not compared against the proposed setup. Including such a comparison would clarify where the NP transfer setting stand
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Algorithms · Imbalanced Data Classification Techniques · Domain Adaptation and Few-Shot Learning