Bucketed PCA Neural Networks with Neurons Mirroring Signals

TL;DR

This paper introduces a bucketed PCA neural network that applies PCA to data buckets, creating a neural architecture that is easier to understand and can achieve accuracy comparable to deep neural networks, especially in supervised classification tasks.

Contribution

The paper presents a novel bucketed PCA neural network architecture that benchmarks DNNs and offers improved interpretability by mirroring input signals.

Findings

Achieves up to 96% accuracy on MNIST with the PCA-NN.

Simpler conceptual building blocks compared to DNNs.

PCA neurons mirror input signals, enhancing interpretability.

Abstract

The bucketed PCA neural network (PCA-NN) with transforms is developed here in an effort to benchmark deep neural networks (DNN's), for problems on supervised classification. Most classical PCA models apply PCA to the entire training data set to establish a reductive representation and then employ non-network tools such as high-order polynomial classifiers. In contrast, the bucketed PCA-NN applies PCA to individual buckets which are constructed in two consecutive phases, as well as retains a genuine architecture of a neural network. This facilitates a fair apple-to-apple comparison to DNN's, esp. to reveal that a major chunk of accuracy achieved by many impressive DNN's could possibly be explained by the bucketed PCA-NN (e.g., 96% out of 98% for the MNIST data set as an example). Compared with most DNN's, the three building blocks of the bucketed PCA-NN are easier to comprehend conceptually - PCA, transforms, and bucketing for error correction. Furthermore, unlike the somewhat quasi-random neurons ubiquitously observed in DNN's, the PCA neurons resemble or mirror the input signals and are more straightforward to decipher as a result.