Issues in Stacked Generalization

TL;DR

This paper investigates key issues in stacked generalization, revealing that combining confidence scores yields better results and demonstrating its effectiveness across various classifiers compared to other ensemble methods.

Contribution

It identifies the importance of using confidence scores in the higher-level model and empirically evaluates stacked generalization's performance with different classifiers.

Findings

Confidence-based combining improves accuracy

Stacked generalization outperforms majority vote

Effective across multiple classifier types

Abstract

Stacked generalization is a general method of using a high-level model to combine lower-level models to achieve greater predictive accuracy. In this paper we address two crucial issues which have been considered to be a `black art' in classification tasks ever since the introduction of stacked generalization in 1992 by Wolpert: the type of generalizer that is suitable to derive the higher-level model, and the kind of attributes that should be used as its input. We find that best results are obtained when the higher-level model combines the confidence (and not just the predictions) of the lower-level ones. We demonstrate the effectiveness of stacked generalization for combining three different types of learning algorithms for classification tasks. We also compare the performance of stacked generalization with majority vote and published results of arcing and bagging.