Boosting Network Weight Separability via Feed-Backward Reconstruction

TL;DR

This paper introduces a novel metric and a boosting method to enhance weight separability in neural networks, leading to improved performance in image classification and face recognition tasks.

Contribution

It proposes a new evaluation metric based on semi-orthogonality and Frobenius distance, along with a feed-backward reconstruction loss to explicitly boost weight separability.

Findings

Improved recognition accuracy on image classification tasks.

Enhanced face recognition performance.

Universal applicability across visual recognition tasks.

Abstract

This paper proposes a new evaluation metric and boosting method for weight separability in neural network design. In contrast to general visual recognition methods designed to encourage both intra-class compactness and inter-class separability of latent features, we focus on estimating linear independence of column vectors in weight matrix and improving the separability of weight vectors. To this end, we propose an evaluation metric for weight separability based on semi-orthogonality of a matrix and Frobenius distance, and the feed-backward reconstruction loss which explicitly encourages weight separability between the column vectors in the weight matrix. The experimental results on image classification and face recognition demonstrate that the weight separability boosting via minimization of feed-backward reconstruction loss can improve the visual recognition performance, hence universally boosting the performance on various visual recognition tasks.