Rule Mining for Correcting Classification Models

TL;DR

This paper introduces a correction rule mining approach to identify and rectify inaccurate predictions in machine learning models, especially useful for models integrated into complex systems, by discovering rules that describe problematic input subpopulations.

Contribution

It proposes a novel correction rule mining algorithm combining frequent itemset mining with a pruning technique to efficiently find rules for correcting model inaccuracies.

Findings

The algorithm effectively identifies subpopulations with inaccurate predictions.

It helps in collecting data insufficiently learned by the model.

The rules enable direct correction of model outputs and analysis of concept drift.

Abstract

Machine learning models need to be continually updated or corrected to ensure that the prediction accuracy remains consistently high. In this study, we consider scenarios where developers should be careful to change the prediction results by the model correction, such as when the model is part of a complex system or software. In such scenarios, the developers want to control the specification of the corrections. To achieve this, the developers need to understand which subpopulations of the inputs get inaccurate predictions by the model. Therefore, we propose correction rule mining to acquire a comprehensive list of rules that describe inaccurate subpopulations and how to correct them. We also develop an efficient correction rule mining algorithm that is a combination of frequent itemset mining and a unique pruning technique for correction rules. We observed that the proposed algorithm found various rules which help to collect data insufficiently learned, directly correct model outputs, and analyze concept drift.