Heuristic Ternary Error-Correcting Output Codes Via Weight Optimization and Layered Clustering-Based Approach

TL;DR

This paper introduces WOLC-ECOC, a novel heuristic method for designing error-correcting output codes that improves multiclass classification by iterative clustering and optimized decoding, showing strong results on multiple datasets.

Contribution

It proposes a new heuristic ECOC framework with layered clustering and optimized decoding, enhancing classifier ensemble performance.

Findings

Effective on 14 UCI datasets and music genre classification

Reduces training risk while maintaining small code length

Improves multiclass classification accuracy

Abstract

One important classifier ensemble for multiclass classification problems is Error-Correcting Output Codes (ECOCs). It bridges multiclass problems and binary-class classifiers by decomposing multiclass problems to a serial binary-class problems. In this paper, we present a heuristic ternary code, named Weight Optimization and Layered Clustering-based ECOC (WOLC-ECOC). It starts with an arbitrary valid ECOC and iterates the following two steps until the training risk converges. The first step, named Layered Clustering based ECOC (LC-ECOC), constructs multiple strong classifiers on the most confusing binary-class problem. The second step adds the new classifiers to ECOC by a novel Optimized Weighted (OW) decoding algorithm, where the optimization problem of the decoding is solved by the cutting plane algorithm. Technically, LC-ECOC makes the heuristic training process not blocked by some difficult binary-class problem. OW decoding guarantees the non-increase of the training risk for ensuring a small code length. Results on 14 UCI datasets and a music genre classification problem demonstrate the effectiveness of WOLC-ECOC.