Interpreting Shared Deep Learning Models via Explicable Boundary Trees

TL;DR

This paper introduces a method to interpret complex deep learning models by constructing a boundary tree from a small, privacy-preserving subset of training data, enhancing transparency and trust in model sharing.

Contribution

The paper proposes a novel approach to interpret deep models using boundary trees built from limited training data, improving understanding without sharing full datasets.

Findings

Boundary trees approximate complex models with high fidelity.

Traversing the tree improves user understanding of model decisions.

Method enhances trust in shared models under privacy constraints.

Abstract

Despite outperforming the human in many tasks, deep neural network models are also criticized for the lack of transparency and interpretability in decision making. The opaqueness results in uncertainty and low confidence when deploying such a model in model sharing scenarios, when the model is developed by a third party. For a supervised machine learning model, sharing training process including training data provides an effective way to gain trust and to better understand model predictions. However, it is not always possible to share all training data due to privacy and policy constraints. In this paper, we propose a method to disclose a small set of training data that is just sufficient for users to get the insight of a complicated model. The method constructs a boundary tree using selected training data and the tree is able to approximate the complicated model with high fidelity. We show that traversing data points in the tree gives users significantly better understanding of the model and paves the way for trustworthy model sharing.