DIVERSE: Disagreement-Inducing Vector Evolution for Rashomon Set Exploration

TL;DR

DIVERSE is a framework that efficiently explores the Rashomon set of neural networks by generating diverse high-performing models without retraining, using feature modulation and evolutionary strategies.

Contribution

It introduces a novel method combining FiLM layers and CMA-ES to systematically explore model diversity within the Rashomon set without retraining.

Findings

DIVERSE uncovers multiple functionally distinct models with high accuracy.

The method achieves diversity comparable to retraining-based approaches.

It reduces computational costs while exploring model multiplicity.

Abstract

We propose DIVERSE, a framework for systematically exploring the Rashomon set of deep neural networks, the collection of models that match a reference model's accuracy while differing in their predictive behavior. DIVERSE augments a pretrained model with Feature-wise Linear Modulation (FiLM) layers and uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search a latent modulation space, generating diverse model variants without retraining or gradient access. Across MNIST, PneumoniaMNIST, and CIFAR-10, DIVERSE uncovers multiple high-performing yet functionally distinct models. Our experiments show that DIVERSE offers a competitive and efficient exploration of the Rashomon set, making it feasible to construct diverse sets that maintain robustness and performance while supporting well-balanced model multiplicity. While retraining remains the baseline to generate Rashomon sets, DIVERSE achieves comparable diversity at reduced computational cost.