Embracing Biased Transition Matrices for Complementary-Label Learning with Many Classes

TL;DR

This paper introduces a novel biased label generation approach in complementary-label learning, significantly improving scalability and accuracy in large multi-class settings like CIFAR-100 and TinyImageNet-200.

Contribution

Proposes Bias-Induced Constrained Labeling (BICL), a framework that leverages deliberate bias in label generation to enable effective learning with many classes.

Findings

Achieves over sevenfold accuracy improvements on CIFAR-100 and TinyImageNet-200.

Demonstrates that non-uniform label generation overcomes traditional scalability limitations.

Establishes a new direction for scalable complementary-label learning.

Abstract

Complementary-label learning (CLL) is a weakly supervised paradigm where instances are labeled with classes they do not belong to. Despite a decade of research, CLL methods remain competitive mainly on 10-class classification, with scaling to large label spaces continuing to be an enduring bottleneck. This limitation stems from the common assumption of uniform label generation in traditional methods, which fatally dilutes the learning signal in many-class settings. In this paper, we demonstrate that this long-standing barrier can be overcome by deliberately designing a biased (non-uniform) generation process that restricts complementary labels to a subset of classes. This finding motivates us to propose Bias-Induced Constrained Labeling (BICL), a principled framework spanning data collection to training that leverages this bias. BICL enables effective learning on CIFAR-100 and TinyImageNet-200, achieving more than sevenfold accuracy improvements over traditional methods. Our findings establish a new trajectory for making CLL feasible for many classes in real-world applications.