'Almost Sure' Chaotic Properties of Machine Learning Methods

TL;DR

This paper argues that machine learning methods inherently exhibit chaotic behavior due to their nature as fixed point iterations in computable function space, explaining some counterintuitive properties of deep learning.

Contribution

It establishes that chaos is a universal property of machine learning algorithms, providing a theoretical foundation for understanding their unpredictable behaviors.

Findings

Machine learning methods are 'almost surely' chaotic.

Chaotic properties are universal across learning algorithms.

Explains counterintuitive behaviors in deep learning.

Abstract

It has been demonstrated earlier that universal computation is 'almost surely' chaotic. Machine learning is a form of computational fixed point iteration, iterating over the computable function space. We showcase some properties of this iteration, and establish in general that the iteration is 'almost surely' of chaotic nature. This theory explains the observation in the counter intuitive properties of deep learning methods. This paper demonstrates that these properties are going to be universal to any learning method.