Machine Collaboration

TL;DR

This paper introduces Machine Collaboration (MaC), a novel ensemble learning framework that employs circular and interactive mechanisms among base models, leading to improved prediction accuracy over traditional methods.

Contribution

MaC is a new ensemble framework that enables circular information transfer among models, with theoretical risk bounds and superior empirical performance.

Findings

MaC outperforms state-of-the-art methods on most datasets.

Theoretical risk bounds support the effectiveness of MaC.

Experimental results show significant accuracy improvements.

Abstract

We propose a new ensemble framework for supervised learning, called machine collaboration (MaC), using a collection of base machines for prediction tasks. Unlike bagging/stacking (a parallel & independent framework) and boosting (a sequential & top-down framework), MaC is a type of circular & interactive learning framework. The circular & interactive feature helps the base machines to transfer information circularly and update their structures and parameters accordingly. The theoretical result on the risk bound of the estimator from MaC reveals that the circular & interactive feature can help MaC reduce risk via a parsimonious ensemble. We conduct extensive experiments on MaC using both simulated data and 119 benchmark real datasets. The results demonstrate that in most cases, MaC performs significantly better than several other state-of-the-art methods, including classification and regression trees, neural networks, stacking, and boosting.