Interpretable Diversity Analysis: Visualizing Feature Representations In Low-Cost Ensembles

TL;DR

This paper introduces interpretability methods to analyze and visualize the differences in feature representations within low-cost neural network ensembles, providing new insights beyond traditional output-based diversity measures.

Contribution

It presents novel interpretability techniques for qualitative analysis of diversity in feature representations of ensemble models, especially for low-cost ensemble algorithms.

Findings

Feature representations differ significantly between ensemble members.

Visualization reveals how diversity evolves during training.

Insights can improve ensemble design and diversity promotion.

Abstract

Diversity is an important consideration in the construction of robust neural network ensembles. A collection of well trained models will generalize better if they are diverse in the patterns they respond to and the predictions they make. Diversity is especially important for low-cost ensemble methods because members often share network structure in order to avoid training several independent models from scratch. Diversity is traditionally analyzed by measuring differences between the outputs of models. However, this gives little insight into how knowledge representations differ between ensemble members. This paper introduces several interpretability methods that can be used to qualitatively analyze diversity. We demonstrate these techniques by comparing the diversity of feature representations between child networks using two low-cost ensemble algorithms, Snapshot Ensembles and Prune and Tune Ensembles. We use the same pre-trained parent network as a starting point for both methods which allows us to explore how feature representations evolve over time. This approach to diversity analysis can lead to valuable insights and new perspectives for how we measure and promote diversity in ensemble methods.