ICX360: In-Context eXplainability 360 Toolkit

TL;DR

ICX360 is an open-source toolkit designed to explain large language models by analyzing their prompts and outputs, using both perturbation and gradient-based methods, to improve transparency in high-stakes applications.

Contribution

The paper introduces ICX360, a comprehensive toolkit for explaining LLM outputs with new implementations of recent explainability methods focused on user prompts.

Findings

Provides tools for black-box and white-box explanations

Supports use cases like retrieval augmented generation and jailbreaking

Includes tutorials and guidance for practical application

Abstract

Large Language Models (LLMs) have become ubiquitous in everyday life and are entering higher-stakes applications ranging from summarizing meeting transcripts to answering doctors' questions. As was the case with earlier predictive models, it is crucial that we develop tools for explaining the output of LLMs, be it a summary, list, response to a question, etc. With these needs in mind, we introduce In-Context Explainability 360 (ICX360), an open-source Python toolkit for explaining LLMs with a focus on the user-provided context (or prompts in general) that are fed to the LLMs. ICX360 contains implementations for three recent tools that explain LLMs using both black-box and white-box methods (via perturbations and gradients respectively). The toolkit, available at https://github.com/IBM/ICX360, contains quick-start guidance materials as well as detailed tutorials covering use cases such as retrieval augmented generation, natural language generation, and jailbreaking.