A Novel ECOC Algorithm with Centroid Distance Based Soft Coding Scheme

TL;DR

This paper introduces CSECOC, a novel ECOC algorithm employing a centroid distance-based soft coding scheme that captures class distribution tendencies, leading to improved classification accuracy with small ensembles.

Contribution

It proposes the first soft coding scheme for ECOC, utilizing class distribution tendencies and regression classifiers, enhancing classification performance over traditional hard coding methods.

Findings

Achieves comparable or better accuracy than state-of-the-art ECOC algorithms.

Uses regression as base learners instead of classifiers.

Demonstrates effectiveness on five UCI datasets.

Abstract

In ECOC framework, the ternary coding strategy is widely deployed in coding process. It relabels classes with {"-1,0,1" }, where -1/1 means to assign the corresponding classes to the negative/positive group, and label 0 leads to ignore the corresponding classes in the training process. However, the application of hard labels may lose some information about the tendency of class distributions. Instead, we propose a Centroid distance-based Soft coding scheme to indicate such tendency, named as CSECOC. In our algorithm, Sequential Forward Floating Selection (SFFS) is applied to search an optimal class assignment by minimizing the ratio of intra-group and inter-group distance. In this way, a hard coding matrix is generated initially. Then we propose a measure, named as coverage, to describe the probability of a sample in a class falling to a correct group. The coverage of a class a group replace the corresponding hard element, so as to form a soft coding matrix. Compared with the hard ones, such soft elements can reflect the tendency of a class belonging to positive or negative group. Instead of classifiers, regressors are used as base learners in this algorithm. To the best of our knowledge, it is the first time that soft coding scheme has been proposed. The results on five UCI datasets show that compared with some state-of-art ECOC algorithms, our algorithm can produce comparable or better classification accuracy with small scale ensembles.