Dirichlet-Based Coarse-to-Fine Example Selection For Open-Set Annotation

TL;DR

This paper introduces a Dirichlet-based coarse-to-fine example selection strategy for open-set annotation, improving active learning by effectively handling open-set noise and distinguishing known from unknown classes.

Contribution

It proposes a novel Dirichlet-based evidential deep learning approach combined with a two-stage selection strategy to enhance open-set active learning performance.

Findings

Achieves state-of-the-art results on various datasets.

Effectively distinguishes known and unknown classes.

Improves active learning robustness in open-set scenarios.

Abstract

Active learning (AL) has achieved great success by selecting the most valuable examples from unlabeled data. However, they usually deteriorate in real scenarios where open-set noise gets involved, which is studied as open-set annotation (OSA). In this paper, we owe the deterioration to the unreliable predictions arising from softmax-based translation invariance and propose a Dirichlet-based Coarse-to-Fine Example Selection (DCFS) strategy accordingly. Our method introduces simplex-based evidential deep learning (EDL) to break translation invariance and distinguish known and unknown classes by considering evidence-based data and distribution uncertainty simultaneously. Furthermore, hard known-class examples are identified by model discrepancy generated from two classifier heads, where we amplify and alleviate the model discrepancy respectively for unknown and known classes. Finally, we combine the discrepancy with uncertainties to form a two-stage strategy, selecting the most informative examples from known classes. Extensive experiments on various openness ratio datasets demonstrate that DCFS achieves state-of-art performance.