DEFRAG: Deep Euclidean Feature Representations through Adaptation on the Grassmann Manifold

TL;DR

DEFRAG is a training method for deep networks that produces Euclidean, well-clustered, and separable feature representations by combining a clustering-based loss with Grassmann manifold projection, leading to improved classification performance.

Contribution

It introduces a novel two-step training approach that enforces Euclidean, clustered, and well-separated feature spaces in deep networks, utilizing Grassmann manifold adaptation.

Findings

Achieves state-of-the-art results on standard datasets.

Uses fewer parameters than comparable networks.

Ensures features are Euclidean and strongly clustered.

Abstract

We propose a novel technique for training deep networks with the objective of obtaining feature representations that exist in a Euclidean space and exhibit strong clustering behavior. Our desired features representations have three traits: they can be compared using a standard Euclidian distance metric, samples from the same class are tightly clustered, and samples from different classes are well separated. However, most deep networks do not enforce such feature representations. The DEFRAG training technique consists of two steps: first good feature clustering behavior is encouraged though an auxiliary loss function based on the Silhouette clustering metric. Then the feature space is retracted onto a Grassmann manifold to ensure that the L_2 Norm forms a similarity metric. The DEFRAG technique achieves state of the art results on standard classification datasets using a relatively small network architecture with significantly fewer parameters than many standard networks.