SECOS: Semantic Capture for Rigorous Classification in Open-World Semi-Supervised Learning

TL;DR

SECOS introduces a semantic capture approach for open-world semi-supervised learning, enabling direct and rigorous classification of known and novel classes using external knowledge and semantic alignment.

Contribution

The paper presents SECOS, a novel method that explicitly predicts textual labels from candidate sets, improving semantic correspondence and performance in OWSSL tasks.

Findings

SECOS surpasses existing OWSSL methods by up to 5.4% in accuracy.

SECOS effectively extracts and aligns semantic representations across modalities.

SECOS performs well even under more lenient evaluation settings.

Abstract

In open-world semi-supervised learning (OWSSL), a model learns from labeled data and unlabeled data containing both known and novel classes. In practical OWSSL applications, models are expected to perform rigorous classification by directly selecting the most semantically relevant label from a candidate set for each sample. Existing OWSSL methods fail to achieve this because novel samples are trained without explicit supervision, and these methods lack mechanisms to extract latent semantic information, resulting in predicted labels that have no semantic correspondence to candidate textual labels. To address this, we introduce SEmantic Capture for Open-world Semi-supervised learning (SECOS), which directly predicts textual labels from the candidate set without post-processing, meeting the requirements of practical OWSSL applications. SECOS leverages external knowledge to extract and align semantic representations across modalities for both known and novel classes, providing explicit supervisory signals for training novel classes. Extensive experiments demonstrate that even when existing OWSSL methods are evaluated under the more lenient post-hoc matching setting, SECOS still surpasses them by up to 5.4\% without such assistance, highlighting its superior effectiveness. Code is available at https://github.com/ganchi-huanggua/OSSL-Classification.