RuleMatrix: Visualizing and Understanding Classifiers with Rules

TL;DR

RuleMatrix is an interactive visualization tool that helps non-expert users understand, explore, and validate machine learning models by representing them as rule-based matrices, enhancing interpretability and transparency.

Contribution

The paper introduces RuleMatrix, a novel matrix-based visualization technique that makes black-box classifiers understandable for users with limited machine learning knowledge.

Findings

Effective in helping users understand model behavior

Facilitates validation of rules and model predictions

Usability study confirms its usefulness for non-experts

Abstract

With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. We design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.