Kalman Filter-based Heuristic Ensemble (KFHE): A new perspective on multi-class ensemble classification using Kalman filters

TL;DR

This paper presents KFHE, a novel multi-class ensemble classification method that models ensemble training as a state estimation problem using Kalman filters, demonstrating superior robustness and accuracy.

Contribution

Introduces a new perspective on ensemble classification as a state estimation problem and develops the KFHE algorithm based on Kalman filters.

Findings

KFHE outperforms state-of-the-art algorithms on multiple datasets.

KFHE maintains high accuracy with noisy class labels.

The new perspective improves ensemble robustness and effectiveness.

Abstract

This paper introduces a new perspective on multi-class ensemble classification that considers training an ensemble as a state estimation problem. The new perspective considers the final ensemble classifier model as a static state, which can be estimated using a Kalman filter that combines noisy estimates made by individual classifier models. A new algorithm based on this perspective, the Kalman Filter-based Heuristic Ensemble (KFHE), is also presented in this paper which shows the practical applicability of the new perspective. Experiments performed on 30 datasets compare KFHE with state-of-the-art multi-class ensemble classification algorithms and show the potential and effectiveness of the new perspective and algorithm. Existing ensemble approaches trade off classification accuracy against robustness to class label noise, but KFHE is shown to be significantly better or at least as good as the state-of-the-art algorithms for datasets both with and without class label noise.