Probability-driven scoring functions in combining linear classifiers

TL;DR

This paper introduces a probability-driven scoring function for combining linear classifiers in ensemble systems, leveraging both measurement and geometrical spaces to potentially improve classification performance.

Contribution

The paper proposes a novel fusion method based on a probability-driven scoring function that depends on the orientation of decision hyperplanes in linear classifier ensembles.

Findings

Some performance improvements observed under specific conditions.

The method compares favorably with existing approaches on benchmark datasets.

Statistical analysis supports the effectiveness of the proposed fusion scheme.

Abstract

Although linear classifiers are one of the oldest methods in machine learning, they are still very popular in the machine learning community. This is due to their low computational complexity and robustness to overfitting. Consequently, linear classifiers are often used as base classifiers of multiple ensemble classification systems. This research is aimed at building a new fusion method dedicated to the ensemble of linear classifiers. The fusion scheme uses both measurement space and geometrical space. Namely, we proposed a probability-driven scoring function which shape depends on the orientation of the decision hyperplanes generated by the base classifiers. The proposed fusion method is compared with the reference method using multiple benchmark datasets taken from the KEEL repository. The comparison is done using multiple quality criteria. The statistical analysis of the obtained results is also performed. The experimental study shows that, under certain conditions, some improvement may be obtained.