Attributions Beyond Neural Networks: The Linear Program Case

TL;DR

This paper explores how attribution methods from explainable AI can be applied to neural encodings of linear programs, revealing insights into high-dimensional solutions and proposing criteria to compare attribution techniques.

Contribution

It introduces neural encodings for LPs that enable the application of XAI attribution methods and analyzes their mathematical properties and empirical behaviors.

Findings

Saliency and LIME produce similar results under perturbation.

Directedness distinguishes attribution methods based on feature increase relevance.

Baseline selection impacts Integrated Gradients beyond computer vision.

Abstract

Linear Programs (LPs) have been one of the building blocks in machine learning and have championed recent strides in differentiable optimizers for learning systems. While there exist solvers for even high-dimensional LPs, understanding said high-dimensional solutions poses an orthogonal and unresolved problem. We introduce an approach where we consider neural encodings for LPs that justify the application of attribution methods from explainable artificial intelligence (XAI) designed for neural learning systems. The several encoding functions we propose take into account aspects such as feasibility of the decision space, the cost attached to each input, or the distance to special points of interest. We investigate the mathematical consequences of several XAI methods on said neural LP encodings. We empirically show that the attribution methods Saliency and LIME reveal indistinguishable results up to perturbation levels, and we propose the property of Directedness as the main discriminative criterion between Saliency and LIME on one hand, and a perturbation-based Feature Permutation approach on the other hand. Directedness indicates whether an attribution method gives feature attributions with respect to an increase of that feature. We further notice the baseline selection problem beyond the classical computer vision setting for Integrated Gradients.