Cluster-based classification with neural ODEs via control

TL;DR

This paper introduces a cluster-based control strategy for neural ODEs in binary classification, reducing control complexity by steering clusters instead of individual points, with theoretical and numerical support for high-dimensional advantages.

Contribution

It proposes a novel cluster-based control method for neural ODE classification that reduces control switches and analyzes its effectiveness in high-dimensional spaces.

Findings

Cluster control reduces complexity from O(N) to O(N/d).

High-dimensional spaces increase the probability of successful classification with constant controls.

Numerical experiments support the optimality of the cluster-based approach.

Abstract

We address binary classification using neural ordinary differential equations from the perspective of simultaneous control of $N$ data points. We consider a single-neuron architecture with parameters fixed as piecewise constant functions of time. In this setting, the model complexity can be quantified by the number of control switches. Previous work has shown that classification can be achieved using a point-by-point strategy that requires $O(N)$ switches. We propose a new control method that classifies any arbitrary dataset by sequentially steering clusters of $d$ points, thereby reducing the complexity to $O(N/d)$ switches. The optimality of this result, particularly in high dimensions, is supported by some numerical experiments. Our complexity bound is sufficient but often conservative because same-class points tend to appear in larger clusters, simplifying classification. This motivates studying the probability distribution of the number of switches required. We introduce a simple control method that imposes a collinearity constraint on the parameters, and analyze a worst-case scenario where both classes have the same size and all points are i.i.d. Our results highlight the benefits of high-dimensional spaces, showing that classification using constant controls becomes more probable as $d$ increases.