A Generalization Method of Partitioned Activation Function for Complex Number

TL;DR

This paper introduces a method to extend real partitioned activation functions into complex numbers, enabling the development of complex neural networks with potential for holomorphic properties and better handling of complex data.

Contribution

It proposes a novel generalization technique for partitioned activation functions from real to complex numbers, including four variations with different properties.

Findings

Applied to LReLU and SELU as examples

Potential to achieve holomorphic activation functions

Conserves complex angle and interaction between real and imaginary parts

Abstract

A method to convert real number partitioned activation function into complex number one is provided. The method has 4em variations; 1 has potential to get holomorphic activation, 2 has potential to conserve complex angle, and the last 1 guarantees interaction between real and imaginary parts. The method has been applied to LReLU and SELU as examples. The complex number activation function is an building block of complex number ANN, which has potential to properly deal with complex number problems. But the complex activation is not well established yet. Therefore, we propose a way to extend the partitioned real activation to complex number.