PReLU: Yet Another Single-Layer Solution to the XOR Problem

TL;DR

This paper shows that a single-layer neural network with PReLU activation can solve the XOR problem, challenging the common belief that multiple layers are necessary, and compares its performance to other methods.

Contribution

It introduces the use of PReLU activation in a single-layer network to solve XOR, highlighting its simplicity and effectiveness.

Findings

Single-layer PReLU network achieves 100% success in XOR

Uses only three learnable parameters

Performs well across various learning rates

Abstract

This paper demonstrates that a single-layer neural network using Parametric Rectified Linear Unit (PReLU) activation can solve the XOR problem, a simple fact that has been overlooked so far. We compare this solution to the multi-layer perceptron (MLP) and the Growing Cosine Unit (GCU) activation function and explain why PReLU enables this capability. Our results show that the single-layer PReLU network can achieve 100\% success rate in a wider range of learning rates while using only three learnable parameters.