Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation

TL;DR

This paper explores how the Jacobian norm relates to open-set recognition, revealing that inter-class separation enhances the detection of unknown classes, and introduces a new loss function to improve OSR performance.

Contribution

The paper provides a theoretical analysis linking Jacobian norm dynamics to OSR and proposes a novel m-OvR loss to improve inter-class separation for better unknown class detection.

Findings

Jacobian norm decreases within known classes due to intra-class learning

Jacobian norm increases for unknown classes due to inter-class learning

The proposed m-OvR loss improves open-set recognition accuracy

Abstract

The findings on open-set recognition (OSR) show that models trained on classification datasets are capable of detecting unknown classes not encountered during the training process. Specifically, after training, the learned representations of known classes dissociate from the representations of the unknown class, facilitating OSR. In this paper, we investigate this emergent phenomenon by examining the relationship between the Jacobian norm of representations and the inter/intra-class learning dynamics. We provide a theoretical analysis, demonstrating that intra-class learning reduces the Jacobian norm for known class samples, while inter-class learning increases the Jacobian norm for unknown samples, even in the absence of direct exposure to any unknown sample. Overall, the discrepancy in the Jacobian norm between the known and unknown classes enables OSR. Based on this insight, which highlights the pivotal role of inter-class learning, we devise a marginal one-vs-rest (m-OvR) loss function that promotes strong inter-class separation. To further improve OSR performance, we integrate the m-OvR loss with additional strategies that maximize the Jacobian norm disparity. We present comprehensive experimental results that support our theoretical observations and demonstrate the efficacy of our proposed OSR approach.