Agreement-based Learning

TL;DR

This paper introduces an agreement-based learning framework that trains multiple models to agree on predictions, improving performance and robustness, especially with unlabeled data, inspired by human learning principles.

Contribution

It proposes a novel agreement-based training framework and algorithm that outperforms existing methods and leverages unlabeled data for better results.

Findings

The proposed framework outperforms traditional model selection methods.

Performance improves with more unlabeled data.

The approach is inspired by human learning processes.

Abstract

Model selection is a problem that has occupied machine learning researchers for a long time. Recently, its importance has become evident through applications in deep learning. We propose an agreement-based learning framework that prevents many of the pitfalls associated with model selection. It relies on coupling the training of multiple models by encouraging them to agree on their predictions while training. In contrast with other model selection and combination approaches used in machine learning, the proposed framework is inspired by human learning. We also propose a learning algorithm defined within this framework which manages to significantly outperform alternatives in practice, and whose performance improves further with the availability of unlabeled data. Finally, we describe a number of potential directions for developing more flexible agreement-based learning algorithms.