Creating Powerful and Interpretable Models with Regression Networks

TL;DR

This paper introduces Regression Networks, a novel neural network architecture that balances high predictive power with interpretability, outperforming existing interpretable models and approaching the performance of dense neural networks.

Contribution

The paper presents Regression Networks, a new architecture that incorporates interactions for improved interpretability without sacrificing neural network power.

Findings

Outperforms state-of-the-art interpretable models on benchmarks.

Matches the performance of dense neural networks.

Can be extended to convolutional and recurrent architectures.

Abstract

As the discipline has evolved, research in machine learning has been focused more and more on creating more powerful neural networks, without regard for the interpretability of these networks. Such "black-box models" yield state-of-the-art results, but we cannot understand why they make a particular decision or prediction. Sometimes this is acceptable, but often it is not. We propose a novel architecture, Regression Networks, which combines the power of neural networks with the understandability of regression analysis. While some methods for combining these exist in the literature, our architecture generalizes these approaches by taking interactions into account, offering the power of a dense neural network without forsaking interpretability. We demonstrate that the models exceed the state-of-the-art performance of interpretable models on several benchmark datasets, matching the power of a dense neural network. Finally, we discuss how these techniques can be generalized to other neural architectures, such as convolutional and recurrent neural networks.