Set Visualizations for Comparing and Evaluating Machine Learning Models

TL;DR

This paper introduces set visualizations and an interactive system, SetMLVis, to improve the comparison and evaluation of machine learning models, especially object detection models, by providing clearer, more interpretable visual comparisons.

Contribution

It presents a novel set visualization approach for model comparison and introduces SetMLVis, an interactive tool that enhances model evaluation efficiency and interpretability.

Findings

SetMLVis improves task completion rates.

SetMLVis reduces cognitive workload.

SetMLVis outperforms traditional visualization methods.

Abstract

Machine learning practitioners often need to compare multiple models to select the best one for their application. However, current methods of comparing models fall short because they rely on aggregate metrics that can be difficult to interpret or do not provide enough information to understand the differences between models. To better support the comparison of models, we propose set visualizations of model outputs to enable easier model-to-model comparison. We outline the requirements for using sets to compare machine learning models and demonstrate how this approach can be applied to various machine learning tasks. We also introduce SetMLVis, an interactive system that utilizes set visualizations to compare object detection models. Our evaluation shows that SetMLVis outperforms traditional visualization techniques in terms of task completion and reduces cognitive workload for users. Supplemental materials can be found at https://osf.io/afksu/?view_only=bb7f259426ad425f81d0518a38c597be.