Collapsing of dimensionality

TL;DR

This paper introduces a novel online learning approach that modifies regularization networks by incorporating time, enabling a dynamic, evolving model that captures input regularities and simplifies the learning process.

Contribution

It presents a new method for online learning based on a time-regularized approach that creates a dynamic graph structure to store input regularities, advancing regularization network techniques.

Findings

Effective in capturing input regularities over time

Demonstrates promising results on artificial datasets

Provides insights into parameter behavior in the proposed model

Abstract

We analyze a new approach to Machine Learning coming from a modification of classical regularization networks by casting the process in the time dimension, leading to a sort of collapse of dimensionality in the problem of learning the model parameters. This approach allows the definition of a online learning algorithm that progressively accumulates the knowledge provided in the input trajectory. The regularization principle leads to a solution based on a dynamical system that is paired with a procedure to develop a graph structure that stores the input regularities acquired from the temporal evolution. We report an extensive experimental exploration on the behavior of the parameter of the proposed model and an evaluation on artificial dataset.