Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles

TL;DR

This paper introduces ArchExplorer, a visual analysis tool that helps researchers understand neural architecture spaces by clustering and visualizing architectures based on structural distances, thereby revealing design principles and improving architecture selection.

Contribution

We develop a novel visual analysis method that efficiently computes and visualizes neural architecture relationships, aiding in understanding and summarizing design principles.

Findings

Effective clustering of architectures based on structural distances

Visualization reveals global and local architecture relationships

Assists in selecting better-performing neural architectures

Abstract

Recent advances in artificial intelligence largely benefit from better neural network architectures. These architectures are a product of a costly process of trial-and-error. To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles. The key idea behind our method is to make the architecture space explainable by exploiting structural distances between architectures. We formulate the pairwise distance calculation as solving an all-pairs shortest path problem. To improve efficiency, we decompose this problem into a set of single-source shortest path problems. The time complexity is reduced from O(kn^2N) to O(knN). Architectures are hierarchically clustered according to the distances between them. A circle-packing-based architecture visualization has been developed to convey both the global relationships between clusters and local neighborhoods of the architectures in each cluster. Two case studies and a post-analysis are presented to demonstrate the effectiveness of ArchExplorer in summarizing design principles and selecting better-performing architectures.