ArcGate: Adaptive Arctangent Gated Activation

TL;DR

ArcGate introduces a flexible, learnable activation function with seven parameters per layer, optimizing non-linearity for improved performance and robustness in remote sensing neural networks.

Contribution

It proposes a novel adaptive activation function, ArcGate, that outperforms fixed-shape activations and adapts to data and network depth.

Findings

ArcGate achieves 99.67% accuracy on PatternNet.

It maintains a 26.65% performance advantage over ReLU under Gaussian noise.

Learned parameters show depth-dependent functional evolution.

Abstract

Activation functions are central to deep networks, influencing non-linearity, feature learning, convergence, and robustness. This paper proposes the Adaptive Arctangent Gated Activation (ArcGate) function, a flexible formulation that generates a broad spectrum of activation shapes via a three-stage non-linear transformation. Unlike conventional fixed-shape activations such as ReLU, GELU, or SiLU, ArcGate uses seven learnable parameters per layer, allowing the neural network to autonomously optimize its non-linearity to the specific requirements of the feature hierarchy and data distribution. We evaluate ArcGate using ResNet-50 and Vision Transformer (ViT-B/16) architectures on three widely used remote sensing benchmarks: PatternNet, UC Merced Land Use, and the 13-band EuroSAT MSI multispectral dataset. Experimental results show that ArcGate consistently outperforms standard baselines, achieving a peak overall accuracy of 99.67% on PatternNet. Most notably, ArcGate exhibits superior structural resilience in noisy environments, maintaining a 26.65% performance lead over ReLU under moderate Gaussian noise (standard deviation 0.1). Analysis of the learned parameters reveals a depth-dependent functional evolution, where the model increases gating strength in deeper layers to enhance signal propagation. These findings suggest that ArcGate is a robust and adaptive general node activation function for high-resolution earth observation tasks.