LORD: Leveraging Open-Set Recognition with Unknown Data

TL;DR

LORD introduces a framework that explicitly models open space during classifier training to improve open-set recognition, effectively leveraging unknown data and background data strategies, including mixup, for better detection of out-of-distribution samples.

Contribution

The paper proposes LORD, a novel open-set recognition framework that explicitly models open space and systematically evaluates training strategies, including the use of mixup to reduce background data dependency.

Findings

LORD improves recognition of unknown data across benchmarks.

Explicit open space modeling enhances open-set recognition performance.

Mixup effectively substitutes background datasets, maintaining high OSR accuracy.

Abstract

Handling entirely unknown data is a challenge for any deployed classifier. Classification models are typically trained on a static pre-defined dataset and are kept in the dark for the open unassigned feature space. As a result, they struggle to deal with out-of-distribution data during inference. Addressing this task on the class-level is termed open-set recognition (OSR). However, most OSR methods are inherently limited, as they train closed-set classifiers and only adapt the downstream predictions to OSR. This work presents LORD, a framework to Leverage Open-set Recognition by exploiting unknown Data. LORD explicitly models open space during classifier training and provides a systematic evaluation for such approaches. We identify three model-agnostic training strategies that exploit background data and applied them to well-established classifiers. Due to LORD's extensive evaluation protocol, we consistently demonstrate improved recognition of unknown data. The benchmarks facilitate in-depth analysis across various requirement levels. To mitigate dependency on extensive and costly background datasets, we explore mixup as an off-the-shelf data generation technique. Our experiments highlight mixup's effectiveness as a substitute for background datasets. Lightweight constraints on mixup synthesis further improve OSR performance.