Improving Fine-Tuning with Latent Cluster Correction

TL;DR

This paper introduces a fine-tuning method that enhances neural network performance by optimizing latent semantic clusters using community detection and a custom clustering loss, demonstrated on CIFAR-100.

Contribution

It proposes a novel fine-tuning approach that leverages latent cluster correction with community detection to improve classification accuracy.

Findings

Latent clusters correlate with final accuracy during training.

The method improves performance on classical neural networks.

Preliminary results show viability on CIFAR-100.

Abstract

The existence of salient semantic clusters in the latent spaces of a neural network during training strongly correlates its final accuracy on classification tasks. This paper proposes a novel fine-tuning method that boosts performance by optimising the formation of these latent clusters, using the Louvain community detection algorithm and a specifically designed clustering loss function. We present preliminary results that demonstrate the viability of this process on classical neural network architectures during fine-tuning on the CIFAR-100 dataset.