Towards a Mathematical Theory of Abstraction

TL;DR

This paper develops a mathematical framework for understanding and learning abstractions of complex systems, defining them as summaries that answer specific queries and measuring their leakiness to optimize their accuracy from data.

Contribution

It introduces a formal definition of abstraction as a set of summaries capable of answering queries, and proposes a method to learn these abstractions directly from data.

Findings

Defines abstraction as summaries answering system queries

Introduces leakiness as a measure of abstraction accuracy

Suggests a data-driven approach to learn abstractions

Abstract

While the utility of well-chosen abstractions for understanding and predicting the behaviour of complex systems is well appreciated, precisely what an abstraction $\textit{is}$ has so far has largely eluded mathematical formalization. In this paper, we aim to set out a mathematical theory of abstraction. We provide a precise characterisation of what an abstraction is and, perhaps more importantly, suggest how abstractions can be learnt directly from data both for static datasets and for dynamical systems. We define an abstraction to be a small set of `summaries' of a system which can be used to answer a set of queries about the system or its behaviour. The difference between the ground truth behaviour of the system on the queries and the behaviour of the system predicted only by the abstraction provides a measure of the `leakiness' of the abstraction which can be used as a loss function to directly learn abstractions from data. Our approach can be considered a generalization of classical statistics where we are not interested in reconstructing `the data' in full, but are instead only concerned with answering a set of arbitrary queries about the data. While highly theoretical, our results have deep implications for statistical inference and machine learning and could be used to develop explicit methods for learning precise kinds of abstractions directly from data.