Neural Surveillance: Live-Update Visualization of Latent Training Dynamics

TL;DR

SentryCam is a real-time visualization tool that monitors the evolution of hidden representations in neural networks during training, aiding in understanding, auditing, and early detection of training issues.

Contribution

This paper introduces SentryCam, a novel live-update visualization framework for internal neural network states, validated across multiple architectures and datasets, with practical auditing capabilities.

Findings

SentryCam provides high-fidelity, low-latency visualizations of training dynamics.

Automated geometry-based alerts can detect training instability up to 7 epochs early.

The framework is adaptable for both exploratory analysis and proactive model auditing.

Abstract

Monitoring the inner state of deep neural networks is essential for auditing the learning process and enabling timely interventions. While conventional metrics like validation loss offer a surface-level view of performance, the evolution of a model's hidden representations provides a deeper, complementary window into its internal dynamics. However, the literature lacks a real-time tool to monitor these crucial internal states. To address this, we introduce SentryCam, a live-update visualization framework that tracks the progression of hidden representations throughout training. SentryCam produces high-fidelity visualizations of the evolving representation space with minimal latency, serving as a powerful dashboard for understanding how a model learns. We quantitatively validate the faithfulness of SentryCam's visualizations across diverse datasets and architectures (ResNet, ViT). Furthermore, we demonstrate SentryCam's practical utility for model auditing through a case study on training instability. We designed an automated auditing system with geometry-based alerts that successfully identified impending model failure up to 7 epochs earlier than was evident from the validation loss curve. SentryCam's flexible framework is easily adaptable, supporting both the exploratory analysis and proactive auditing essential for robust model development. The code is available at https://github.com/xianglinyang/SentryCam.