A Visual Tool for Interactive Model Explanation using Sensitivity Analysis

TL;DR

SAInT is an interactive Python tool that combines local and global sensitivity analysis to help users understand ML models through visual explanations, supporting human-in-the-loop workflows without coding.

Contribution

The paper introduces SAInT, a novel visual tool that integrates sensitivity analysis with interactive model explanation, enabling non-programmers to explore and refine ML models effectively.

Findings

Demonstrated on Titanic dataset for survival prediction.

Showed sensitivity analysis guides feature selection.

Enabled model understanding without programming expertise.

Abstract

We present SAInT, a Python-based tool for visually exploring and understanding the behavior of Machine Learning (ML) models through integrated local and global sensitivity analysis. Our system supports Human-in-the-Loop (HITL) workflows by enabling users - both AI researchers and domain experts - to configure, train, evaluate, and explain models through an interactive graphical interface without programming. The tool automates model training and selection, provides global feature attribution using variance-based sensitivity analysis, and offers per-instance explanation via LIME and SHAP. We demonstrate the system on a classification task predicting survival on the Titanic dataset and show how sensitivity information can guide feature selection and data refinement.