Towards Label-Efficient Incremental Learning: A Survey

TL;DR

This survey reviews recent advances in label-efficient incremental learning, emphasizing semi-supervised, few-shot, and self-supervised methods to reduce labeling needs and improve scalability in real-world applications.

Contribution

It is the first comprehensive survey focusing on label-efficient incremental learning, identifying key subdivisions and future directions for enhancing scalability and reducing labeling efforts.

Findings

Highlighting the importance of semi-, few-shot, and self-supervised learning in incremental learning.

Identifying challenges and potential solutions for label-efficient incremental learning.

Proposing future research directions to improve scalability and reduce labeling costs.

Abstract

The current dominant paradigm when building a machine learning model is to iterate over a dataset over and over until convergence. Such an approach is non-incremental, as it assumes access to all images of all categories at once. However, for many applications, non-incremental learning is unrealistic. To that end, researchers study incremental learning, where a learner is required to adapt to an incoming stream of data with a varying distribution while preventing forgetting of past knowledge. Significant progress has been made, however, the vast majority of works focus on the fully supervised setting, making these algorithms label-hungry thus limiting their real-life deployment. To that end, in this paper, we make the first attempt to survey recently growing interest in label-efficient incremental learning. We identify three subdivisions, namely semi-, few-shot- and self-supervised learning to reduce labeling efforts. Finally, we identify novel directions that can further enhance label-efficiency and improve incremental learning scalability. Project website: https://github.com/kilickaya/label-efficient-il.