Teraflop-scale Incremental Machine Learning

TL;DR

This paper introduces a long-term memory architecture for artificial general intelligence using Solomonoff's incremental learning, a Scheme-based reference machine, and a Levin Search variant with update algorithms, demonstrating effective incremental learning.

Contribution

It presents a novel long-term memory design for AGI based on incremental learning with a new Levin Search variant and multiple update algorithms, advancing the field of scalable AI systems.

Findings

Effective incremental learning demonstrated in experiments

Successful integration of multiple update algorithms

Scalable approach suitable for long-term AGI development

Abstract

We propose a long-term memory design for artificial general intelligence based on Solomonoff's incremental machine learning methods. We use R5RS Scheme and its standard library with a few omissions as the reference machine. We introduce a Levin Search variant based on Stochastic Context Free Grammar together with four synergistic update algorithms that use the same grammar as a guiding probability distribution of programs. The update algorithms include adjusting production probabilities, re-using previous solutions, learning programming idioms and discovery of frequent subprograms. Experiments with two training sequences demonstrate that our approach to incremental learning is effective.