Connecting Sphere Manifolds Hierarchically for Regularization

TL;DR

This paper introduces a hierarchical sphere manifold regularization technique for classification that enhances neural network performance by structuring class classifiers on interconnected spheres aligned with class hierarchies.

Contribution

It proposes a novel regularization method connecting class classifiers on sphere manifolds based on hierarchy, replacing the final neural network layer.

Findings

Improves ResNet and DenseNet accuracy on multiple datasets

Effective hierarchical regularization for deep networks

Enhances class separation through sphere manifold connections

Abstract

This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).