# Exact and Consistent Interpretation of Piecewise Linear Models Hidden behind APIs: A Closed Form Solution

## TL;DR

This paper introduces OpenAPI, a closed form solution for exactly and consistently interpreting predictions of hidden piecewise linear models behind APIs, overcoming limitations of heuristic probing methods.

## Contribution

The paper presents a novel closed form approach for exact and consistent interpretation of piecewise linear models behind APIs, ensuring reliable explanations without access to model parameters.

## Key findings

- OpenAPI provides exact interpretations of model predictions.
- The method guarantees consistency across different inputs.
- Experiments validate the accuracy and reliability of the approach.

## Abstract

More and more AI services are provided through APIs on cloud where predictive models are hidden behind APIs. To build trust with users and reduce potential application risk, it is important to interpret how such predictive models hidden behind APIs make their decisions. The biggest challenge of interpreting such predictions is that no access to model parameters or training data is available. Existing works interpret the predictions of a model hidden behind an API by heuristically probing the response of the API with perturbed input instances. However, these methods do not provide any guarantee on the exactness and consistency of their interpretations. In this paper, we propose an elegant closed form solution named OpenAPI to compute exact and consistent interpretations for the family of Piecewise Linear Models (PLM), which includes many popular classification models. The major idea is to first construct a set of overdetermined linear equation systems with a small set of perturbed instances and the predictions made by the model on those instances. Then, we solve the equation systems to identify the decision features that are responsible for the prediction on an input instance. Our extensive experiments clearly demonstrate the exactness and consistency of our method.