The Representational Status of Deep Learning Models

TL;DR

This paper examines the nature of representations in deep learning models, arguing they encode global relational information rather than local, decomposable features, impacting explainability and future AI research.

Contribution

It clarifies the ambiguous concept of 'representation' in DLMs, emphasizing their global relational encoding over local semantic decomposition.

Findings

DLMs primarily encode global relational information.

Local, decomposable representations are not generally present.

Implications for explainable AI and scientific understanding.

Abstract

This paper aims to clarify the representational status of Deep Learning Models (DLMs). While commonly referred to as 'representations', what this entails is ambiguous due to a conflation of functional and relational conceptions of representation. This paper argues that while DLMs represent their targets in a relational sense, in general, we have no good reason to believe that DLMs encode locally semantically decomposable representations of their targets. That is, the representational capacity these models have is largely global, rather than decomposable into stable, local subrepresentations. This result has immediate implications for explainable AI (XAI) and directs attention toward exploring the global relational nature of deep learning representations and their relationship both to models more generally to understand their potential role in future scientific inquiry.