NullSpaceNet: Nullspace Convoluional Neural Network with Differentiable Loss Function

TL;DR

NullSpaceNet introduces a novel neural network that maps inputs to a interpretable nullspace, achieving higher accuracy, fewer parameters, and faster inference compared to traditional CNNs across multiple datasets.

Contribution

The paper presents NullSpaceNet with a new nullspace mapping and a differentiable loss function, significantly improving interpretability, accuracy, and efficiency over existing CNNs.

Findings

Achieves up to 4.55% higher accuracy

Reduces parameters from 135M to 19M

Cuts inference time by 99%

Abstract

We propose NullSpaceNet, a novel network that maps from the pixel level input to a joint-nullspace (as opposed to the traditional feature space), where the newly learned joint-nullspace features have clearer interpretation and are more separable. NullSpaceNet ensures that all inputs from the same class are collapsed into one point in this new joint-nullspace, and the different classes are collapsed into different points with high separation margins. Moreover, a novel differentiable loss function is proposed that has a closed-form solution with no free-parameters. NullSpaceNet exhibits superior performance when tested against VGG16 with fully-connected layer over 4 different datasets, with accuracy gain of up to 4.55%, a reduction in learnable parameters from 135M to 19M, and reduction in inference time of 99% in favor of NullSpaceNet. This means that NullSpaceNet needs less than 1% of the time it takes a traditional CNN to classify a batch of images with better accuracy.