HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning

TL;DR

HEX introduces a human-in-the-loop deep reinforcement learning framework for machine learning explainability, focusing on decision boundary understanding and functioning effectively with limited or federated data.

Contribution

It presents a novel HEX approach that synthesizes explanation policies considering the decision boundary, addressing limited data and trust issues in MLX.

Findings

Effective in limited data scenarios

Captures decision boundary explicitly

Operates with federated learning settings

Abstract

The use of machine learning (ML) models in decision-making contexts, particularly those used in high-stakes decision-making, are fraught with issue and peril since a person - not a machine - must ultimately be held accountable for the consequences of the decisions made using such systems. Machine learning explainability (MLX) promises to provide decision-makers with prediction-specific rationale, assuring them that the model-elicited predictions are made for the right reasons and are thus reliable. Few works explicitly consider this key human-in-the-loop (HITL) component, however. In this work we propose HEX, a human-in-the-loop deep reinforcement learning approach to MLX. HEX incorporates 0-distrust projection to synthesize decider specific explanation-providing policies from any arbitrary classification model. HEX is also constructed to operate in limited or reduced training data scenarios, such as those employing federated learning. Our formulation explicitly considers the decision boundary of the ML model in question, rather than the underlying training data, which is a shortcoming of many model-agnostic MLX methods. Our proposed methods thus synthesize HITL MLX policies that explicitly capture the decision boundary of the model in question for use in limited data scenarios.