Taylor Outlier Exposure

TL;DR

Taylor Outlier Exposure (TaylorOE) introduces a polynomial regularization approach that enables effective training on noisy OOD datasets contaminated with in-distribution samples, improving out-of-distribution detection performance.

Contribution

The paper proposes TaylorOE, a novel regularization method using Taylor expansion to control regularization strength, allowing training on contaminated OOD datasets for better OOD detection.

Findings

TaylorOE outperforms conventional methods on OOD detection tasks.

The regularization control via Taylor expansion improves robustness to noisy OOD data.

Experimental results confirm the effectiveness of the proposed approach.

Abstract

Out-of-distribution (OOD) detection is the task of identifying data sampled from distributions that were not used during training. This task is essential for reliable machine learning and a better understanding of their generalization capabilities. Among OOD detection methods, Outlier Exposure (OE) significantly enhances OOD detection performance and generalization ability by exposing auxiliary OOD data to the model. However, constructing clean auxiliary OOD datasets, uncontaminated by in-distribution (ID) samples, is essential for OE; generally, a noisy OOD dataset contaminated with ID samples negatively impacts OE training dynamics and final detection performance. Furthermore, as dataset scale increases, constructing clean OOD data becomes increasingly challenging and costly. To address these challenges, we propose Taylor Outlier Exposure (TaylorOE), an OE-based approach with regularization that allows training on noisy OOD datasets contaminated with ID samples. Specifically, we represent the OE regularization term as a polynomial function via a Taylor expansion, allowing us to control the regularization strength for ID data in the auxiliary OOD dataset by adjusting the order of Taylor expansion. In our experiments on the OOD detection task with clean and noisy OOD datasets, we demonstrate that the proposed method consistently outperforms conventional methods and analyze our regularization term to show its effectiveness. Our implementation code of TaylorOE is available at \url{https://github.com/fukuchan41/TaylorOE}.