First Power Linear Unit with Sign

TL;DR

This paper introduces FPLUS, a novel activation function based on power functions with sign, demonstrating superior performance and stability across CNN architectures and extending it to a more flexible generalized form called PFPLUS.

Contribution

The paper presents the first power-based activation function with sign, including a generalized version with learnable parameters, validated through theoretical derivation and extensive experiments.

Findings

FPLUS outperforms existing activation functions on benchmark datasets.

FPLUS maintains stability across various CNN architectures.

PFPLUS enhances expressive capacity with learnable parameters.

Abstract

This paper proposes a novel and insightful activation method termed FPLUS, which exploits mathematical power function with polar signs in form. It is enlightened by common inverse operation while endowed with an intuitive meaning of bionics. The formulation is derived theoretically under conditions of some prior knowledge and anticipative properties, and then its feasibility is verified through a series of experiments using typical benchmark datasets, whose results indicate our approach owns superior competitiveness among numerous activation functions, as well as compatible stability across many CNN architectures. Furthermore, we extend the function presented to a more generalized type called PFPLUS with two parameters that can be fixed or learnable, so as to augment its expressive capacity, and outcomes of identical tests validate this improvement.