APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation

TL;DR

The paper introduces the APTx Neuron, a unified trainable neuron architecture that combines activation and computation, leading to more efficient and expressive neural networks validated on MNIST.

Contribution

It presents the APTx Neuron, a novel unified neuron design that integrates activation and linear transformation into a single trainable unit, simplifying architecture.

Findings

Achieved 96.69% accuracy on MNIST within 11 epochs.

Reduced parameter count to approximately 332K.

Demonstrated superior expressiveness and training efficiency.

Abstract

We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the APTx activation function, thereby eliminating the need for separate activation layers and making the architecture both optimization-efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((\alpha_i + \tanh(\beta_i x_i)) \cdot \gamma_i x_i) + \delta$, where all parameters $\alpha_i$, $\beta_i$, $\gamma_i$, and $\delta$ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to $96.69\%$ test accuracy within 11 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and training efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it. Source code is available at https://github.com/mr-ravin/aptx_neuron.