Training Experimentally Robust and Interpretable Binarized Regression Models Using Mixed-Integer Programming

TL;DR

This paper presents a mixed-integer programming approach to train robust, interpretable binarized regression models for multiclass classification, outperforming traditional methods especially on corrupted datasets.

Contribution

Introduces a novel MIP-based training method that balances margin optimization and model simplicity for robust, interpretable binarized regression models.

Findings

MIP model outperforms PBO and is competitive with LR and GD on standard datasets.

MIP model achieves superior accuracy on corrupted datasets.

Demonstrates interpretability of learned parameters on MNIST.

Abstract

In this paper, we explore model-based approach to training robust and interpretable binarized regression models for multiclass classification tasks using Mixed-Integer Programming (MIP). Our MIP model balances the optimization of prediction margin and model size by using a weighted objective that: minimizes the total margin of incorrectly classified training instances, maximizes the total margin of correctly classified training instances, and maximizes the overall model regularization. We conduct two sets of experiments to test the classification accuracy of our MIP model over standard and corrupted versions of multiple classification datasets, respectively. In the first set of experiments, we show that our MIP model outperforms an equivalent Pseudo-Boolean Optimization (PBO) model and achieves competitive results to Logistic Regression (LR) and Gradient Descent (GD) in terms of classification accuracy over the standard datasets. In the second set of experiments, we show that our MIP model outperforms the other models (i.e., GD and LR) in terms of classification accuracy over majority of the corrupted datasets. Finally, we visually demonstrate the interpretability of our MIP model in terms of its learned parameters over the MNIST dataset. Overall, we show the effectiveness of training robust and interpretable binarized regression models using MIP.