Towards Realistic Long-tailed Semi-supervised Learning in an Open World
Yuanpeng He, Lijian Li
TL;DR
This paper introduces ROLSSL, a realistic open-world long-tailed semi-supervised learning setting that does not assume distribution similarity between known and novel categories, and proposes a dual-stage logit adjustment method to improve learning with imbalanced data.
Contribution
The paper defines a new ROLSSL setting that reflects real-world challenges and proposes a dual-stage logit adjustment approach to handle distribution biases in semi-supervised learning.
Findings
Achieved up to 50.1% performance improvement on CIFAR100 and ImageNet100.
Effectively mitigates category bias in imbalanced semi-supervised learning.
Establishes a strong baseline for open-world long-tailed semi-supervised learning.
Abstract
Open-world long-tailed semi-supervised learning (OLSSL) has increasingly attracted attention. However, existing OLSSL algorithms generally assume that the distributions between known and novel categories are nearly identical. Against this backdrop, we construct a more \emph{Realistic Open-world Long-tailed Semi-supervised Learning} (\textbf{ROLSSL}) setting where there is no premise on the distribution relationships between known and novel categories. Furthermore, even within the known categories, the number of labeled samples is significantly smaller than that of the unlabeled samples, as acquiring valid annotations is often prohibitively costly in the real world. Under the proposed ROLSSL setting, we propose a simple yet potentially effective solution called dual-stage post-hoc logit adjustments. The proposed approach revisits the logit adjustment strategy by considering the…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
1. The combination of long-tailed, semi-supervised, and open-world learning is interesting.
1. The writing of this paper is not very clear. For example, 'S/N Consistency' in Tab. 1, a key feature of the proposed setting, is never explained. 2. Open-world Long-tailed Semi-supervised Learning seems quite similar to [*1], which is not discussed in related works. Therefore, the novelty of this paper may be weak. 3. Experiments may be insufficient. Only some tiny datasets are considered. Large-scale benchmarks (e.g., ImageNet-LT or iNaturalist) should be included. [*1] SimPro: A Simple Pr
1. This paper is well-motivated and easy to follow. 2. The proposed problem setting is realistic and reflects the long-tailed nature of the real world. 3. This paper conducts comprehensive experiments to show the superiority of the method.
1. The comparative methods are outdated, and the most recent comparative method of this paper is ECCV 2022. In the literature, OSSL is also referred to as generalized category discovery (GCD). Some recent GCD methods, including GCD [R1] and SimGCD [R2], should be included in Table 2 and 3. There are also some works [R3, R4] in GCD considering long-tailed scenarios, which are highly related to this work and should be compared. 2. The novelty of this paper is limited. The basic loss functions $L_{
1. The problem addressed by this paper is highly practical, as determining the distributional relationships between known and novel categories is indeed nearly impossible in real-world scenarios. 2. The paper is well-written and easy to follow. 3. Key experimental results demonstrate that the DPLA approach consistently outperforms the OpenLDN baseline in both known and novel class recognition, achieving up to a 50.1% improvement on datasets like CIFAR-100 and ImageNet-100. Authors further show
1. A key concern is the complexity of the proposed pipeline, which involves multiple stages and numerous hyper-parameters. With additional stages and hyper-parameters to tune, it becomes more challenging to generalize this pipeline to new problem settings and datasets. For instance, as shown in Tables 3 and 4, using non-optimal scaling factors can significantly impact performance on CIFAR-100 and ImageNet-100, despite both datasets having the same number of classes. Have you explored methods to
SSL in a more realistic setting is proposed and explored in this work.
1. Since readers may not be quite familiar with the development of open-world long-tail SSL, it is suggested that more background about the existing OLSSL algorithms be included in Section 3.2, especially for the definition of Eq(1). 2. The dual-stage post-hoc logit adjustment section is not easy to read. Specifically, - why the aremax is used in Eq(2)? According to the Fig 1, it is expected to be a distribution adjustment. - What is the definition of F_{y_i^l} in Eq(3)? If the labeled samples
1. The studied problem in this paper is practical and underexplored. Most previous studies focus on long-tailed or open-world semi-supervised learning, without considering both. 2. The proposed approach is simple and easy to implement. This paper improves previous methods by proposing a dual-stage post-hoc logit adjustment technique, which does not involve complex strategies or additional learnable parameters. 3. Experiments on six datasets show the advantage of the proposed approach against pre
1. The novelty of the proposed method seems limited. As stated in the *Strengths*, the proposed method is simple and easy to implement because it is pretty much built upon existing techniques. However, this also makes the contribution of the method minor. 2. The rationale behind the method is not well-explained. It is unclear why the logit adjustment strategies for labeled and unlabeled data are different. Also, the design of Eq. (3) is not straightforward and needs in-depth understanding and j
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsText and Document Classification Technologies · Machine Learning and Data Classification · Anomaly Detection Techniques and Applications