Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective

TL;DR

This paper presents a novel perspective on logistic regression and neural networks by interpreting their outputs as Dempster-Shafer mass functions, enhancing interpretability and decision-making under uncertainty.

Contribution

It introduces a Dempster-Shafer framework for logistic regression and neural networks, providing new insights into feature roles and alternative decision rules.

Findings

Mass functions offer more information than probabilities.

Features' contributions can be interpreted through mass functions.

Interval dominance allows for set-based class decisions.

Abstract

We revisit logistic regression and its nonlinear extensions, including multilayer feedforward neural networks, by showing that these classifiers can be viewed as converting input or higher-level features into Dempster-Shafer mass functions and aggregating them by Dempster's rule of combination. The probabilistic outputs of these classifiers are the normalized plausibilities corresponding to the underlying combined mass function. This mass function is more informative than the output probability distribution. In particular, it makes it possible to distinguish between lack of evidence (when none of the features provides discriminant information) from conflicting evidence (when different features support different classes). This expressivity of mass functions allows us to gain insight into the role played by each input feature in logistic regression, and to interpret hidden unit outputs in multilayer neural networks. It also makes it possible to use alternative decision rules, such as interval dominance, which select a set of classes when the available evidence does not unambiguously point to a single class, thus trading reduced error rate for higher imprecision.