DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

TL;DR

DendroMap introduces a treemap-based visualization tool for interactively exploring large-scale image datasets in machine learning, enabling hierarchical organization, detailed inspection, and better understanding of dataset diversity and model performance.

Contribution

This paper presents DendroMap, a novel treemap-based visualization method that improves the exploration of large image datasets by organizing images hierarchically based on high-dimensional features.

Findings

Users can discover dataset insights and model errors effectively.

DendroMap outperforms t-SNE grid visualizations in user preference and task efficiency.

Participants preferred DendroMap for grouping and searching tasks.

Abstract

In this paper, we present DendroMap, a novel approach to interactively exploring large-scale image datasets for machine learning (ML). ML practitioners often explore image datasets by generating a grid of images or projecting high-dimensional representations of images into 2-D using dimensionality reduction techniques (e.g., t-SNE). However, neither approach effectively scales to large datasets because images are ineffectively organized and interactions are insufficiently supported. To address these challenges, we develop DendroMap by adapting Treemaps, a well-known visualization technique. DendroMap effectively organizes images by extracting hierarchical cluster structures from high-dimensional representations of images. It enables users to make sense of the overall distributions of datasets and interactively zoom into specific areas of interests at multiple levels of abstraction. Our case studies with widely-used image datasets for deep learning demonstrate that users can discover insights about datasets and trained models by examining the diversity of images, identifying underperforming subgroups, and analyzing classification errors. We conducted a user study that evaluates the effectiveness of DendroMap in grouping and searching tasks by comparing it with a gridified version of t-SNE and found that participants preferred DendroMap. DendroMap is available at https://div-lab.github.io/dendromap/.