xai_evals : A Framework for Evaluating Post-Hoc Local Explanation Methods

TL;DR

xai_evals is an open-source Python framework that systematically evaluates post-hoc explanation methods for machine learning models, enhancing interpretability and trust in AI systems across various data types.

Contribution

It introduces a comprehensive, standardized toolkit for benchmarking and assessing the reliability of explanation methods like SHAP, LIME, and Grad-CAM.

Findings

Provides evaluation metrics such as faithfulness, sensitivity, and robustness.

Supports multiple data modalities including tabular and image data.

Facilitates transparent comparison of explanation techniques.

Abstract

The growing complexity of machine learning and deep learning models has led to an increased reliance on opaque "black box" systems, making it difficult to understand the rationale behind predictions. This lack of transparency is particularly challenging in high-stakes applications where interpretability is as important as accuracy. Post-hoc explanation methods are commonly used to interpret these models, but they are seldom rigorously evaluated, raising concerns about their reliability. The Python package xai_evals addresses this by providing a comprehensive framework for generating, benchmarking, and evaluating explanation methods across both tabular and image data modalities. It integrates popular techniques like SHAP, LIME, Grad-CAM, Integrated Gradients (IG), and Backtrace, while supporting evaluation metrics such as faithfulness, sensitivity, and robustness. xai_evals enhances the interpretability of machine learning models, fostering transparency and trust in AI systems. The library is open-sourced at https://pypi.org/project/xai-evals/ .