Exploring Cross-model Neuronal Correlations in the Context of Predicting Model Performance and Generalizability

TL;DR

This paper proposes a correlation-based method to assess AI model performance and similarity by comparing neuron outputs across models, aiding trustworthiness and robustness evaluation.

Contribution

It introduces a novel neural correlation approach for evaluating model quality and transferability, scalable across architectures and datasets.

Findings

Higher neural correlation correlates with better performance.

Partial layer comparison reveals architectural similarities.

Correlation-based assessment complements traditional evaluation metrics.

Abstract

As Artificial Intelligence (AI) models are increasingly integrated into critical systems, the need for a robust framework to establish the trustworthiness of AI is increasingly paramount. While collaborative efforts have established conceptual foundations for such a framework, there remains a significant gap in developing concrete, technically robust methods for assessing AI model quality and performance. This paper introduces a novel approach for assessing a newly trained model's performance based on another known model by calculating correlation between neural networks. The proposed method evaluates correlations by determining if, for each neuron in one network, there exists a neuron in the other network that produces similar output. This approach has implications for memory efficiency, allowing for the use of smaller networks when high correlation exists between networks of different sizes. Experiments on five fully connected networks and a two layer CNN trained on MNIST family datasets show that higher alignment with the CNN tracks stronger performance and smaller degradation under black box transfer based attacks. On ImageNet pretrained ResNets and DenseNets, partial layer comparisons recover intuitive architectural affinities, indicating that the procedure scales with reasonable approximations. These results support representational alignment as a lightweight compatibility check that complements standard accuracy, calibration, and robustness evaluations and enables early external validation of new models. Code is available at https://github.com/aheldis/Cross-model-Correlation.git.