Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks

TL;DR

This paper proposes a unified, modular framework for evaluating model representations beyond traditional downstream tasks, focusing on attributes like invariance, equivariance, and disentanglement to better understand their qualities.

Contribution

It introduces a standardized protocol for assessing various qualities of representations, enabling comprehensive evaluation across different models and domains.

Findings

Models with similar downstream performance can differ significantly in invariance and disentanglement.

The framework reveals differences in representation qualities not captured by downstream tasks.

Evaluation results suggest new directions for improving model interpretability and robustness.

Abstract

Downstream probing has been the dominant method for evaluating model representations, an important process given the increasing prominence of self-supervised learning and foundation models. However, downstream probing primarily assesses the availability of task-relevant information in the model's latent space, overlooking attributes such as equivariance, invariance, and disentanglement, which contribute to the interpretability, adaptability, and utility of representations in real-world applications. While some attempts have been made to measure these qualities in representations, no unified evaluation framework with modular, generalizable, and interpretable metrics exists. In this paper, we argue for the importance of representation evaluation beyond downstream probing. We introduce a standardized protocol to quantify informativeness, equivariance, invariance, and disentanglement of factors of variation in model representations. We use it to evaluate representations from a variety of models in the image and speech domains using different architectures and pretraining approaches on identified controllable factors of variation. We find that representations from models with similar downstream performance can behave substantially differently with regard to these attributes. This hints that the respective mechanisms underlying their downstream performance are functionally different, prompting new research directions to understand and improve representations.