Improvising the Learning of Neural Networks on Hyperspherical Manifold

TL;DR

This paper explores the use of stereographic projection to transform data onto a hyperspherical manifold, improving neural network performance on image classification tasks by leveraging angular margin losses.

Contribution

It introduces a novel approach of applying stereographic projection to analyze and enhance angular margin losses in hyperspherical neural network representations.

Findings

Improved classification accuracy on CIFAR-10 and CIFAR-100 datasets.

Effective application on malaria blood smear images.

Theoretical and practical validation of decision boundaries on hypersphere.

Abstract

The impact of convolution neural networks (CNNs) in the supervised settings provided tremendous increment in performance. The representations learned from CNN's operated on hyperspherical manifold led to insightful outcomes in face recognition, face identification, and other supervised tasks. A broad range of activation functions were developed with hypersphere intuition which performs superior to softmax in euclidean space. The main motive of this research is to provide insights. First, the stereographic projection is implied to transform data from Euclidean space ($\mathbb{R}^{n}$) to hyperspherical manifold ($\mathbb{S}^{n}$) to analyze the performance of angular margin losses. Secondly, proving theoretically and practically that decision boundaries constructed on hypersphere using stereographic projection obliges the learning of neural networks. Experiments have demonstrated that applying stereographic projection on existing state-of-the-art angular margin objective functions improved performance for standard image classification data sets (CIFAR-10,100). Further, we ran our experiments on malaria-thin blood smear images, resulting in effective outcomes. The code is publicly available at:https://github.com/barulalithb/stereo-angular-margin.