MOSS: Multi-Objective Optimization for Stable Rule Sets

TL;DR

MOSS is a multi-objective optimization framework that constructs stable, interpretable rule sets balancing accuracy, sparsity, and stability, enabling rapid trade-off evaluation and outperforming existing methods.

Contribution

Introduces MOSS, a novel multi-objective optimization approach with a specialized algorithm for stable rule set construction and trade-off analysis.

Findings

MOSS outperforms state-of-the-art rule ensembles in predictive accuracy.

MOSS produces more stable rule sets compared to existing methods.

The specialized algorithm scales to large problem instances.

Abstract

We present MOSS, a multi-objective optimization framework for constructing stable sets of decision rules. MOSS incorporates three important criteria for interpretability: sparsity, accuracy, and stability, into a single multi-objective optimization framework. Importantly, MOSS allows a practitioner to rapidly evaluate the trade-off between accuracy and stability in sparse rule sets in order to select an appropriate model. We develop a specialized cutting plane algorithm in our framework to rapidly compute the Pareto frontier between these two objectives, and our algorithm scales to problem instances beyond the capabilities of commercial optimization solvers. Our experiments show that MOSS outperforms state-of-the-art rule ensembles in terms of both predictive performance and stability.