Visual Analytics for Explainable and Trustworthy Artificial Intelligence

TL;DR

This paper explores how visual analytics can enhance transparency and trust in AI systems by providing interactive visual tools across different stages of AI development and deployment.

Contribution

It defines a design space for visualizations that improve AI transparency and reviews existing VA dashboards supporting various AI pipeline tasks.

Findings

VA solutions foster trust in AI models

Visual dashboards support debugging and refining AI models

Proposed design space guides future VA development for AI

Abstract

Our society increasingly depends on intelligent systems to solve complex problems, ranging from recommender systems suggesting the next movie to watch to AI models assisting in medical diagnoses for hospitalized patients. With the iterative improvement of diagnostic accuracy and efficiency, AI holds significant potential to mitigate medical misdiagnoses by preventing numerous deaths and reducing an economic burden of approximately 450 EUR billion annually. However, a key obstacle to AI adoption lies in the lack of transparency: many automated systems function as "black boxes," providing predictions without revealing the underlying processes. This opacity can hinder experts' ability to trust and rely on AI systems. Visual analytics (VA) provides a compelling solution by combining AI models with interactive visualizations. These specialized charts and graphs empower users to incorporate their domain expertise to refine and improve the models, bridging the gap between AI and human understanding. In this work, we define, categorize, and explore how VA solutions can foster trust across the stages of a typical AI pipeline. We propose a design space for innovative visualizations and present an overview of our previously developed VA dashboards, which support critical tasks within the various pipeline stages, including data processing, feature engineering, hyperparameter tuning, understanding, debugging, refining, and comparing models.