Integer Programming-based Error-Correcting Output Code Design for Robust Classification

TL;DR

This paper introduces an Integer Programming approach to design optimal Error-Correcting Output Codes for multiclass classification, enhancing robustness against adversarial attacks while maintaining high accuracy.

Contribution

It presents a novel IP formulation for ECOC design that guarantees optimality and leverages graph-theoretic structures for tractability, improving robustness and accuracy.

Findings

Codebooks outperform standard methods in nominal accuracy

IP-designed codebooks show robustness to adversarial perturbations

Method achieves optimality guarantees through integer programming

Abstract

Error-Correcting Output Codes (ECOCs) offer a principled approach for combining simple binary classifiers into multiclass classifiers. In this paper, we investigate the problem of designing optimal ECOCs to achieve both nominal and adversarial accuracy using Support Vector Machines (SVMs) and binary deep learning models. In contrast to previous literature, we present an Integer Programming (IP) formulation to design minimal codebooks with desirable error correcting properties. Our work leverages the advances in IP solvers to generate codebooks with optimality guarantees. To achieve tractability, we exploit the underlying graph-theoretic structure of the constraint set in our IP formulation. This enables us to use edge clique covers to substantially reduce the constraint set. Our codebooks achieve a high nominal accuracy relative to standard codebooks (e.g., one-vs-all, one-vs-one, and dense/sparse codes). We also estimate the adversarial accuracy of our ECOC-based classifiers in a white-box setting. Our IP-generated codebooks provide non-trivial robustness to adversarial perturbations even without any adversarial training.