A mini-batch training strategy for deep subspace clustering networks

TL;DR

This paper presents a scalable mini-batch training strategy for deep subspace clustering that uses a memory bank to handle large datasets and introduces a decoder-free contrastive learning framework for efficient representation learning.

Contribution

It introduces a novel mini-batch training method with a memory bank for deep subspace clustering and a decoder-free contrastive learning framework for fine-tuning large pre-trained encoders.

Findings

Achieves comparable performance to full-batch methods.

Outperforms state-of-the-art on COIL100 and ORL datasets.

Enables scalable training for high-resolution images.

Abstract

Mini-batch training is a cornerstone of modern deep learning, offering computational efficiency and scalability for training complex architectures. However, existing deep subspace clustering (DSC) methods, which typically combine an autoencoder with a self-expressive layer, rely on full-batch processing. The bottleneck arises from the self-expressive module, which requires representations of the entire dataset to construct a self-representation coefficient matrix. In this work, we introduce a mini-batch training strategy for DSC by integrating a memory bank that preserves global feature representations. Our approach enables scalable training of deep architectures for subspace clustering with high-resolution images, overcoming previous limitations. Additionally, to efficiently fine-tune large-scale pre-trained encoders for subspace clustering, we propose a decoder-free framework that leverages contrastive learning instead of autoencoding for representation learning. This design not only eliminates the computational overhead of decoder training but also provides competitive performance. Extensive experiments demonstrate that our approach not only achieves performance comparable to full-batch methods, but outperforms other state-of-the-art subspace clustering methods on the COIL100 and ORL datasets by fine-tuning deep networks.