MLDS: A Dataset for Weight-Space Analysis of Neural Networks

TL;DR

This paper introduces MLDS, a dataset of thousands of trained neural networks designed to facilitate weight-space analysis, revealing insights into model relationships and data influence beyond traditional loss metrics.

Contribution

The paper presents MLDS, a large, controlled dataset of neural networks enabling direct weight-space analysis for better evaluation and understanding of neural models.

Findings

Models cluster in weight-space when trained on identical data

Small data changes cause significant divergence in weight-space

Weight-space analysis can complement or surpass loss-based evaluation

Abstract

Neural networks are powerful models that solve a variety of complex real-world problems. However, the stochastic nature of training and large number of parameters in a typical neural model makes them difficult to evaluate via inspection. Research shows this opacity can hide latent undesirable behavior, be it from poorly representative training data or via malicious intent to subvert the behavior of the network, and that this behavior is difficult to detect via traditional indirect evaluation criteria such as loss. Therefore, it is time to explore direct ways to evaluate a trained neural model via its structure and weights. In this paper we present MLDS, a new dataset consisting of thousands of trained neural networks with carefully controlled parameters and generated via a global volunteer-based distributed computing platform. This dataset enables new insights into both model-to-model and model-to-training-data relationships. We use this dataset to show clustering of models in weight-space with identical training data and meaningful divergence in weight-space with even a small change to the training data, suggesting that weight-space analysis is a viable and effective alternative to loss for evaluating neural networks.