Understanding and Designing Complex Systems: Response to "A framework for optimal high-level descriptions in science and engineering---preliminary report"

TL;DR

This paper reviews methods for modeling complex systems, emphasizing computational mechanics and rate-distortion theory, and discusses future directions for automatic discovery of optimal models, while responding to prior critiques.

Contribution

It clarifies misconceptions about state space compression and situates recent methods within the broader context of model reduction techniques for complex systems.

Findings

Computational mechanics identifies minimal predictive models.

Rate-distortion theory aids in approximating complex models.

Future work aims for automatic discovery of optimal models.

Abstract

We recount recent history behind building compact models of nonlinear, complex processes and identifying their relevant macroscopic patterns or "macrostates". We give a synopsis of computational mechanics, predictive rate-distortion theory, and the role of information measures in monitoring model complexity and predictive performance. Computational mechanics provides a method to extract the optimal minimal predictive model for a given process. Rate-distortion theory provides methods for systematically approximating such models. We end by commenting on future prospects for developing a general framework that automatically discovers optimal compact models. As a response to the manuscript cited in the title above, this brief commentary corrects potentially misleading claims about its state space compression method and places it in a broader historical setting.