An Investigation of Interpretability Techniques for Deep Learning in Predictive Process Analytics

TL;DR

This study compares interpretability techniques for deep neural networks and random forests in medical decision-making, demonstrating how local and structured models can enhance understanding of cancer prediction models.

Contribution

The paper introduces novel interpretability methods for deep learning and random forests applied to medical data, including autoencoder-based representations and local linear models.

Findings

Local models improve understanding of predictions.

Autoencoders reveal nonlinear patient clusters.

Features identified align with known cancer indicators.

Abstract

This paper explores interpretability techniques for two of the most successful learning algorithms in medical decision-making literature: deep neural networks and random forests. We applied these algorithms in a real-world medical dataset containing information about patients with cancer, where we learn models that try to predict the type of cancer of the patient, given their set of medical activity records. We explored different algorithms based on neural network architectures using long short term deep neural networks, and random forests. Since there is a growing need to provide decision-makers understandings about the logic of predictions of black boxes, we also explored different techniques that provide interpretations for these classifiers. In one of the techniques, we intercepted some hidden layers of these neural networks and used autoencoders in order to learn what is the representation of the input in the hidden layers. In another, we investigated an interpretable model locally around the random forest's prediction. Results show learning an interpretable model locally around the model's prediction leads to a higher understanding of why the algorithm is making some decision. Use of local and linear model helps identify the features used in prediction of a specific instance or data point. We see certain distinct features used for predictions that provide useful insights about the type of cancer, along with features that do not generalize well. In addition, the structured deep learning approach using autoencoders provided meaningful prediction insights, which resulted in the identification of nonlinear clusters correspondent to the patients' different types of cancer.