q-Neurons: Neuron Activations based on Stochastic Jackson's Derivative Operators

TL;DR

This paper introduces q-neurons, a novel stochastic neuron type based on Jackson's q-derivatives, which improves neural network performance and is easy to implement.

Contribution

It presents a new stochastic neuron model using Jackson's q-derivatives, enhancing neural network scalability and performance.

Findings

Consistent improvement over standard activation functions in training loss

Enhanced testing performance across multiple tasks

Easy integration into existing neural network architectures

Abstract

We propose a new generic type of stochastic neurons, called $q$-neurons, that considers activation functions based on Jackson's $q$-derivatives with stochastic parameters $q$. Our generalization of neural network architectures with $q$-neurons is shown to be both scalable and very easy to implement. We demonstrate experimentally consistently improved performances over state-of-the-art standard activation functions, both on training and testing loss functions.