Verifying Machine Learning Interpretability Requirements through Provenance

TL;DR

This paper proposes a method using ML provenance data to verify interpretability requirements, enabling quantifiable assessment of model transparency and behavior.

Contribution

It introduces a novel approach for verifying interpretability non-functional requirements in ML by leveraging provenance data.

Findings

Provenance data can be used to verify interpretability requirements.

The approach enables quantifiable verification of ML interpretability.

Model transparency is improved through provenance-based verification.

Abstract

Machine Learning (ML) Engineering is a growing field that necessitates an increase in the rigor of ML development. It draws many ideas from software engineering and more specifically, from requirements engineering. Existing literature on ML Engineering defines quality models and Non-Functional Requirements (NFRs) specific to ML, in particular interpretability being one such NFR. However, a major challenge occurs in verifying ML NFRs, including interpretability. Although existing literature defines interpretability in terms of ML, it remains an immeasurable requirement, making it impossible to definitively confirm whether a model meets its interpretability requirement. This paper shows how ML provenance can be used to verify ML interpretability requirements. This work provides an approach for how ML engineers can save various types of model and data provenance to make the model's behavior transparent and interpretable. Saving this data forms the basis of quantifiable Functional Requirements (FRs) whose verification in turn verifies the interpretability NFR. Ultimately, this paper contributes a method to verify interpretability NFRs for ML models.