Tight Rates in Supervised Outlier Transfer Learning

TL;DR

This paper investigates the theoretical limits and potential of transfer learning for outlier detection, showing that even dissimilar sources can be beneficial and that adaptive methods can achieve optimal transfer.

Contribution

It establishes the information-theoretic limits of outlier transfer learning and demonstrates that adaptive procedures can attain these limits without prior knowledge of source-target discrepancy.

Findings

Seemingly dissimilar sources can provide significant information for transfer.

Information-theoretic limits of transfer are characterized under an extended discrepancy measure.

Adaptive procedures can achieve these limits without prior discrepancy knowledge.

Abstract

A critical barrier to learning an accurate decision rule for outlier detection is the scarcity of outlier data. As such, practitioners often turn to the use of similar but imperfect outlier data from which they might transfer information to the target outlier detection task. Despite the recent empirical success of transfer learning approaches in outlier detection, a fundamental understanding of when and how knowledge can be transferred from a source to a target outlier detection task remains elusive. In this work, we adopt the traditional framework of Neyman-Pearson classification -- which formalizes supervised outlier detection -- with the added assumption that one has access to some related but imperfect outlier data. Our main results are as follows: We first determine the information-theoretic limits of the problem under a measure of discrepancy that extends some existing notions from traditional balanced classification; interestingly, unlike in balanced classification, seemingly very dissimilar sources can provide much information about a target, thus resulting in fast transfer. We then show that, in principle, these information-theoretic limits are achievable by adaptive procedures, i.e., procedures with no a priori information on the discrepancy between source and target outlier distributions.