Cognitive Action Laws: The Case of Visual Features

TL;DR

This paper introduces Cognitive Action Laws (CAL), a theoretical framework for perceptual learning that models neural networks as systems minimizing a cognitive action functional, with applications demonstrated in computer vision feature extraction.

Contribution

It develops a novel theoretical approach linking neural network learning to laws of nature via a functional index, leading to new insights and methods for perceptual learning.

Findings

CAL framework models neural networks as systems minimizing cognitive action.

The theory ensures global minima correspond to stationarity conditions.

Experiments demonstrate effectiveness in computer vision feature extraction.

Abstract

This paper proposes a theory for understanding perceptual learning processes within the general framework of laws of nature. Neural networks are regarded as systems whose connections are Lagrangian variables, namely functions depending on time. They are used to minimize the cognitive action, an appropriate functional index that measures the agent interactions with the environment. The cognitive action contains a potential and a kinetic term that nicely resemble the classic formulation of regularization in machine learning. A special choice of the functional index, which leads to forth-order differential equations---Cognitive Action Laws (CAL)---exhibits a structure that mirrors classic formulation of machine learning. In particular, unlike the action of mechanics, the stationarity condition corresponds with the global minimum. Moreover, it is proven that typical asymptotic learning conditions on the weights can coexist with the initialization provided that the system dynamics is driven under a policy referred to as information overloading control. Finally, the theory is experimented for the problem of feature extraction in computer vision.