Compensation Learning

TL;DR

This paper introduces the concept of compensation learning, a new strategy in machine learning that involves explicitly compensating for data or model deficiencies, and provides a taxonomy and new algorithms demonstrating its effectiveness.

Contribution

The paper systematically defines and categorizes compensation learning, revealing its presence in existing algorithms and proposing new robust learning algorithms utilizing this strategy.

Findings

New algorithms improve robustness in image classification and sentiment analysis

Compensation learning enhances performance across various learning scenarios

Existing algorithms can be interpreted as forms of compensation learning

Abstract

Weighting strategy prevails in machine learning. For example, a common approach in robust machine learning is to exert lower weights on samples which are likely to be noisy or quite hard. This study reveals another undiscovered strategy, namely, compensating. Various incarnations of compensating have been utilized but it has not been explicitly revealed. Learning with compensating is called compensation learning and a systematic taxonomy is constructed for it in this study. In our taxonomy, compensation learning is divided on the basis of the compensation targets, directions, inference manners, and granularity levels. Many existing learning algorithms including some classical ones can be viewed or understood at least partially as compensation techniques. Furthermore, a family of new learning algorithms can be obtained by plugging the compensation learning into existing learning algorithms. Specifically, two concrete new learning algorithms are proposed for robust machine learning. Extensive experiments on image classification and text sentiment analysis verify the effectiveness of the two new algorithms. Compensation learning can also be used in other various learning scenarios, such as imbalance learning, clustering, regression, and so on.