Optimisation Is Not What You Need

TL;DR

This paper argues that optimization methods in AI have fundamental flaws like catastrophic forgetting and overfitting, and suggests that developing true artificial cognition requires exploring beyond traditional machine learning approaches.

Contribution

It formally proves the inherent limitations of optimization methods for artificial cognition and demonstrates that world-modelling approaches can overcome these issues.

Findings

Optimization methods are inherently limited by catastrophic forgetting.

World-modelling methods avoid catastrophic forgetting and overfitting.

AI development should explore beyond optimization-based machine learning.

Abstract

The Artificial Intelligence field has focused on developing optimisation methods to solve multiple problems, specifically problems that we thought to be only solvable through cognition. The obtained results have been outstanding, being able to even surpass the Turing Test. However, we have found that these optimisation methods share some fundamental flaws that impede them to become a true artificial cognition. Specifically, the field have identified catastrophic forgetting as a fundamental problem to develop such cognition. This paper formally proves that this problem is inherent to optimisation methods, and as such it will always limit approaches that try to solve the Artificial General Intelligence problem as an optimisation problem. Additionally, it addresses the problem of overfitting and discuss about other smaller problems that optimisation methods pose. Finally, it empirically shows how world-modelling methods avoid suffering from either problem. As a conclusion, the field of Artificial Intelligence needs to look outside the machine learning field to find methods capable of developing an artificial cognition.