Position-wise optimizer: A nature-inspired optimization algorithm

TL;DR

This paper introduces a new nature-inspired optimization algorithm based on biological neural plasticity, aiming to improve neural network training without complex external controls, and demonstrates its effectiveness on multiple datasets.

Contribution

A novel optimization algorithm inspired by neural plasticity that simplifies training processes compared to existing complex models.

Findings

Outperforms gradient descent on tested datasets

Demonstrates biological plausibility and efficiency

Reduces need for external control mechanisms

Abstract

The human nervous system utilizes synaptic plasticity to solve optimization problems. Previous studies have tried to add the plasticity factor to the training process of artificial neural networks, but most of those models require complex external control over the network or complex novel rules. In this manuscript, a novel nature-inspired optimization algorithm is introduced that imitates biological neural plasticity. Furthermore, the model is tested on three datasets and the results are compared with gradient descent optimization.