LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics

TL;DR

LossLens is a visual analytics framework designed to explore and diagnose the global and local structures of loss landscapes in neural networks, aiding understanding of model behavior and architecture effects.

Contribution

The paper introduces LossLens, a novel visual analytics tool that enables multi-scale visualization of loss landscapes for improved model diagnostics.

Findings

LossLens effectively visualizes the impact of residual connections in ResNet-20.

LossLens reveals how physical parameters affect physics-informed neural networks.

The framework enhances understanding of loss landscape structures at multiple scales.

Abstract

Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. We demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem.