Point Cloud Network: An Order of Magnitude Improvement in Linear Layer Parameter Count

TL;DR

This paper presents the Point Cloud Network (PCN), a new linear layer architecture that significantly reduces parameter count while maintaining accuracy, demonstrated through experiments on AlexNet with CIFAR datasets.

Contribution

The paper introduces PCN, a novel linear layer design that drastically reduces parameters and outperforms traditional MLP layers in deep learning models.

Findings

PCN achieves 99.5% parameter reduction in linear layers.

AlexNet-PCN16 maintains comparable accuracy to original AlexNet.

Empirical evidence supports PCN's efficiency over MLP layers.

Abstract

This paper introduces the Point Cloud Network (PCN) architecture, a novel implementation of linear layers in deep learning networks, and provides empirical evidence to advocate for its preference over the Multilayer Perceptron (MLP) in linear layers. We train several models, including the original AlexNet, using both MLP and PCN architectures for direct comparison of linear layers (Krizhevsky et al., 2012). The key results collected are model parameter count and top-1 test accuracy over the CIFAR-10 and CIFAR-100 datasets (Krizhevsky, 2009). AlexNet-PCN16, our PCN equivalent to AlexNet, achieves comparable efficacy (test accuracy) to the original architecture with a 99.5% reduction of parameters in its linear layers. All training is done on cloud RTX 4090 GPUs, leveraging pytorch for model construction and training. Code is provided for anyone to reproduce the trials from this paper.