Exploratory Visual Analysis for Increasing Data Readiness in Artificial Intelligence Projects

TL;DR

This paper explores how visual analysis methods can enhance data readiness in AI projects by mapping data aspects to visualization techniques and addressing distribution shifts, based on practical use cases.

Contribution

It introduces a mapping between data readiness aspects and visual analysis techniques, extending the concept to include task and solution considerations, especially for time-varying data.

Findings

Effective visual analysis techniques improve data readiness levels.

Addressing distribution shifts enhances data quality for AI.

Practical experiences demonstrate the approach's usefulness.

Abstract

We present experiences and lessons learned from increasing data readiness of heterogeneous data for artificial intelligence projects using visual analysis methods. Increasing the data readiness level involves understanding both the data as well as the context in which it is used, which are challenges well suitable to visual analysis. For this purpose, we contribute a mapping between data readiness aspects and visual analysis techniques suitable for different data types. We use the defined mapping to increase data readiness levels in use cases involving time-varying data, including numerical, categorical, and text. In addition to the mapping, we extend the data readiness concept to better take aspects of the task and solution into account and explicitly address distribution shifts during data collection time. We report on our experiences in using the presented visual analysis techniques to aid future artificial intelligence projects in raising the data readiness level.