Confident Sinkhorn Allocation for Pseudo-Labeling
Vu Nguyen, Hisham Husain, Sachin Farfade, Anton van den, Hengel
TL;DR
This paper introduces Confident Sinkhorn Allocation, a novel method for pseudo-labeling in semi-supervised learning that uses optimal transport to select high-confidence samples, improving accuracy over existing methods.
Contribution
It proposes CSA, a new pseudo-labeling approach that leverages optimal transport and confidence scores, and extends PACBayes bounds using Integral Probability Metrics.
Findings
CSA outperforms state-of-the-art pseudo-labeling methods.
The method effectively handles unlabeled data without domain assumptions.
Improved theoretical bounds for ensemble models using IPMs.
Abstract
Semi-supervised learning is a critical tool in reducing machine learning's dependence on labeled data. It has been successfully applied to structured data, such as images and natural language, by exploiting the inherent spatial and semantic structure therein with pretrained models or data augmentation. These methods are not applicable, however, when the data does not have the appropriate structure, or invariances. Due to their simplicity, pseudo-labeling (PL) methods can be widely used without any domain assumptions. However, the greedy mechanism in PL is sensitive to a threshold and can perform poorly if wrong assignments are made due to overconfidence. This paper studies theoretically the role of uncertainty to pseudo-labeling and proposes Confident Sinkhorn Allocation (CSA), which identifies the best pseudo-label allocation via optimal transport to only samples with high confidence…
Peer Reviews
Decision·Submitted to ICLR 2024
1. The authors incorporate uncertainty into the pseudo-labeling generation process and provide a theoretical interpretation. 2. The authors study theoretically the pseudo-labelling process when training on labeled set and predicting unlabeled data using a PAC-Bayes generalization bound.
1. The choice to employ optimal transport methods for pseudo-labeling is not immediately clear, especially given the existence of the method detailed in section 2.2. It would be beneficial if the authors could elucidate on the rationale behind selecting optimal transport over the direct pseudo-labeling approach from section 2.2. 2. The paper employs an ensemble of M models, but it is ambiguous whether the observed improvement in performance is attributed to the ensemble effect or the proposed al
The issue of excessive confidence and sensitivity to thresholds in pseudo labeling (PL) is indeed intriguing. The author conducts an analysis of the uncertainties within PL and offers some insights into this matter.
1. The paper's contribution appears somewhat vague. While it introduces a new pseudo labeling (PL) method, it lacks a clear probabilistic formulation. However, it's worth noting that the author does provide a PAC-Bayes generalization bound in Section 2.4, particularly for ensembling multiple classifiers. It would enhance clarity to explicitly state the individual contributions of various sections within the methodology. 2. There seems to be an inconsistency in the citation format used in the ma
1. This work introduces an efficient algorithm aimed at mitigating uncertainty in pseudo-labeling. It leverages ensemble models to assess the confidence of labeling. Additionally, a comprehensive experimental setup is designed, encompassing not only accuracy comparisons with state-of-the-art algorithms but also evaluations across various dimensions. 2. This work provides a solid mathematical proof for uncertainty analysis in Pseudo-Labeling and extends PAC-Bayes bounds to ensemble models, both o
1. Errors are present in the tables and figures. In Table 1, it is noted that in the comparison of related approaches, FlexMatch should be characterized as non-greedy based on the provided content. Regarding Figure 4, the top red square on the left fails to adequately illustrate the distinctions in assignments. 2. In the section pertaining to the analysis of uncertainty in Pseudo-Labeling (PL), some aspects of the formulation concerning the settings are found to be incomplete. Consequently, this
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLogic, Reasoning, and Knowledge