Stable Attractors for Neural networks classification via Ordinary Differential Equations (SA-nODE)

TL;DR

This paper introduces a novel neural network classification method based on ordinary differential equations with pre-defined stable attractors, demonstrating its potential as a high-performance classifier rooted in dynamical systems theory.

Contribution

It presents a new approach where untrained models are designed with fixed attractors, enabling classification through dynamical convergence, bridging machine learning and dynamical systems.

Findings

Method performs well on toy models and benchmarks.

It demonstrates the feasibility of using stable attractors for classification.

Performance is below state-of-the-art deep learning but shows promise.

Abstract

A novel approach for supervised classification is presented which sits at the intersection of machine learning and dynamical systems theory. At variance with other methodologies that employ ordinary differential equations for classification purposes, the untrained model is a priori constructed to accommodate for a set of pre-assigned stationary stable attractors. Classifying amounts to steer the dynamics towards one of the planted attractors, depending on the specificity of the processed item supplied as an input. Asymptotically the system will hence converge on a specific point of the explored multi-dimensional space, flagging the category of the object to be eventually classified. Working in this context, the inherent ability to perform classification, as acquired ex post by the trained model, is ultimately reflected in the shaped basin of attractions associated to each of the target stable attractors. The performance of the proposed method is here challenged against simple toy models crafted for the purpose, as well as by resorting to well established reference standards. Although this method does not reach the performance of state-of-the-art deep learning algorithms, it illustrates that continuous dynamical systems with closed analytical interaction terms can serve as high-performance classifiers.