An entropy-optimal path to humble AI

TL;DR

This paper introduces a novel entropy-optimizing reformulation of Boltzmann machines that creates more efficient, reliable, and high-performing AI models with lower costs and data requirements, especially useful in climate prediction.

Contribution

It presents a new mathematical framework for entropy-optimized Boltzmann machines that eliminates gradient descent, ensures model uniqueness, and provides confidence measures, improving performance and efficiency.

Findings

Models are more performant and slim compared to state-of-the-art tools.

Descriptor lengths are close to intrinsic complexity bounds.

Effective in climate prediction with minimal data.

Abstract

Progress of AI has led to very successful, but by no means humble models and tools, especially regarding (i) the huge and further exploding costs and resources they demand, and (ii) the over-confidence of these tools with the answers they provide. Here we introduce a novel mathematical framework for a non-equilibrium entropy-optimizing reformulation of Boltzmann machines based on the exact law of total probability and the exact convex polytope representations. We show that it results in the highly-performant, but much cheaper, gradient-descent-free learning framework with mathematically-justified existence and uniqueness criteria, and cheaply-computable confidence/reliability measures for both the model inputs and the outputs. Comparisons to state-of-the-art AI tools in terms of performance, cost and the model descriptor lengths on a broad set of synthetic and real-world problems with varying complexity reveal that the proposed method results in more performant and slim models, with the descriptor lengths being very close to the intrinsic complexity scaling bounds for the underlying problems. Applying this framework to historical climate data results in models with systematically higher prediction skills for the onsets of important La Ni\~na and El Ni\~no climate phenomena, requiring just few years of climate data for training - a small fraction of what is necessary for contemporary climate prediction tools.