Revisiting Non-separable Binary Classification and its Applications in Anomaly Detection

TL;DR

This paper demonstrates that linear classification of XOR is feasible using a novel equality separation paradigm, which is effective for anomaly detection and can be integrated into neural networks.

Contribution

It introduces equality separation as a new approach to binary classification, extending the capabilities of linear classifiers to XOR and anomaly detection tasks.

Findings

Equality separation can classify XOR linearly.

The method effectively detects both seen and unseen anomalies.

The approach integrates smoothly with neural network pipelines.

Abstract

The inability to linearly classify XOR has motivated much of deep learning. We revisit this age-old problem and show that linear classification of XOR is indeed possible. Instead of separating data between halfspaces, we propose a slightly different paradigm, equality separation, that adapts the SVM objective to distinguish data within or outside the margin. Our classifier can then be integrated into neural network pipelines with a smooth approximation. From its properties, we intuit that equality separation is suitable for anomaly detection. To formalize this notion, we introduce closing numbers, a quantitative measure on the capacity for classifiers to form closed decision regions for anomaly detection. Springboarding from this theoretical connection between binary classification and anomaly detection, we test our hypothesis on supervised anomaly detection experiments, showing that equality separation can detect both seen and unseen anomalies.