VAC-CNN: A Visual Analytics System for Comparative Studies of Deep Convolutional Neural Networks

TL;DR

VAC-CNN is a visual analytics system designed to facilitate in-depth comparison and understanding of multiple CNN models, supporting both quantitative and qualitative analysis for machine learning practitioners.

Contribution

The paper introduces VAC-CNN, a novel visual analytics tool that enables comparison of many CNN models simultaneously with interactive visualization and explanation features.

Findings

Supports comparison of tens of CNN models

Enhances understanding through visualization and explanation

Assists novice practitioners in model evaluation

Abstract

The rapid development of Convolutional Neural Networks (CNNs) in recent years has triggered significant breakthroughs in many machine learning (ML) applications. The ability to understand and compare various CNN models available is thus essential. The conventional approach with visualizing each model's quantitative features, such as classification accuracy and computational complexity, is not sufficient for a deeper understanding and comparison of the behaviors of different models. Moreover, most of the existing tools for assessing CNN behaviors only support comparison between two models and lack the flexibility of customizing the analysis tasks according to user needs. This paper presents a visual analytics system, VAC-CNN (Visual Analytics for Comparing CNNs), that supports the in-depth inspection of a single CNN model as well as comparative studies of two or more models. The ability to compare a larger number of (e.g., tens of) models especially distinguishes our system from previous ones. With a carefully designed model visualization and explaining support, VAC-CNN facilitates a highly interactive workflow that promptly presents both quantitative and qualitative information at each analysis stage. We demonstrate VAC-CNN's effectiveness for assisting novice ML practitioners in evaluating and comparing multiple CNN models through two use cases and one preliminary evaluation study using the image classification tasks on the ImageNet dataset.