The Fixed Sub-Center: A Better Way to Capture Data Complexity

TL;DR

The paper introduces Fixed Sub-Center (F-SC), a novel method that creates multiple, fixed class sub-centers to better capture data distribution complexities while reducing memory and computational costs.

Contribution

F-SC generates fixed, discrepant sub-centers for each class, improving data representation and intra-class compactness without increasing training overhead.

Findings

Significantly improves image classification accuracy.

Enhances fine-grained recognition performance.

Reduces memory and computational costs.

Abstract

Treating class with a single center may hardly capture data distribution complexities. Using multiple sub-centers is an alternative way to address this problem. However, highly correlated sub-classes, the classifier's parameters grow linearly with the number of classes, and lack of intra-class compactness are three typical issues that need to be addressed in existing multi-subclass methods. To this end, we propose to use Fixed Sub-Center (F-SC), which allows the model to create more discrepant sub-centers while saving memory and cutting computational costs considerably. The F-SC specifically, first samples a class center Ui for each class from a uniform distribution, and then generates a normal distribution for each class, where the mean is equal to Ui. Finally, the sub-centers are sampled based on the normal distribution corresponding to each class, and the sub-centers are fixed during the training process avoiding the overhead of gradient calculation. Moreover, F-SC penalizes the Euclidean distance between the samples and their corresponding sub-centers, it helps remain intra-compactness. The experimental results show that F-SC significantly improves the accuracy of both image classification and fine-grained recognition tasks.