One-class systems seamlessly fit in the forward-forward algorithm

TL;DR

This paper explores the integration of one-class objective functions with the forward-forward training algorithm, demonstrating potential benefits for online and memory-efficient neural network training.

Contribution

It shows that one-class loss functions can be effectively used with the forward-forward algorithm, eliminating the need for new loss functions in this context.

Findings

One-class functions work well with forward-forward training.

Memory requirements are reduced during training.

Supports seamless online training scenarios.

Abstract

The forward-forward algorithm presents a new method of training neural networks by updating weights during an inference, performing parameter updates for each layer individually. This immediately reduces memory requirements during training and may lead to many more benefits, like seamless online training. This method relies on a loss ("goodness") function that can be evaluated on the activations of each layer, of which can have a varied parameter size, depending on the hyperparamaterization of the network. In the seminal paper, a goodness function was proposed to fill this need; however, if placed in a one-class problem context, one need not pioneer a new loss because these functions can innately handle dynamic network sizes. In this paper, we investigate the performance of deep one-class objective functions when trained in a forward-forward fashion. The code is available at \url{https://github.com/MichaelHopwood/ForwardForwardOneclass}.