A Double Penalty Model for Interpretability

TL;DR

This paper introduces a double penalty model that separates interpretable and non-interpretable parts of a prediction model, aiming to enhance interpretability with minimal loss in accuracy, using a doubly penalized fitting procedure and convergence guarantees.

Contribution

It proposes a novel double penalty approach that explicitly models interpretability, along with theoretical convergence and consistency results, and demonstrates practical application to biological datasets.

Findings

Model achieves high interpretability with minimal performance loss.

Convergence of cyclic coordinate ascent is established.

Effective in microbiome and diabetes trait prediction datasets.

Abstract

Modern statistical learning techniques have often emphasized prediction performance over interpretability, giving rise to "black box" models that may be difficult to understand, and to generalize to other settings. We conceptually divide a prediction model into interpretable and non-interpretable portions, as a means to produce models that are highly interpretable with little loss in performance. Implementation of the model is achieved by considering separability of the interpretable and non-interpretable portions, along with a doubly penalized procedure for model fitting. We specify conditions under which convergence of model estimation can be achieved via cyclic coordinate ascent, and the consistency of model estimation holds. We apply the methods to datasets for microbiome host trait prediction and a diabetes trait, and discuss practical tradeoff diagnostics to select models with high interpretability.