Human-like Clustering with Deep Convolutional Neural Networks

TL;DR

This paper demonstrates that deep convolutional neural networks, inspired by human visual hierarchy, can effectively perform clustering of complex, overlapping shapes, surpassing traditional algorithms in certain scenarios.

Contribution

It introduces a unified approach using CNNs for both classification and clustering, emphasizing hierarchical structures and compositionality for improved performance.

Findings

CNNs trained with noisy labels can cluster overlapping shapes effectively

Hierarchical visual structures are crucial for human-like clustering performance

CNN-based clustering outperforms some state-of-the-art algorithms in complex scenarios

Abstract

Classification and clustering have been studied separately in machine learning and computer vision. Inspired by the recent success of deep learning models in solving various vision problems (e.g., object recognition, semantic segmentation) and the fact that humans serve as the gold standard in assessing clustering algorithms, here, we advocate for a unified treatment of the two problems and suggest that hierarchical frameworks that progressively build complex patterns on top of the simpler ones (e.g., convolutional neural networks) offer a promising solution. We do not dwell much on the learning mechanisms in these frameworks as they are still a matter of debate, with respect to biological constraints. Instead, we emphasize on the compositionality of the real world structures and objects. In particular, we show that CNNs, trained end to end using back propagation with noisy labels, are able to cluster data points belonging to several overlapping shapes, and do so much better than the state of the art algorithms. The main takeaway lesson from our study is that mechanisms of human vision, particularly the hierarchal organization of the visual ventral stream should be taken into account in clustering algorithms (e.g., for learning representations in an unsupervised manner or with minimum supervision) to reach human level clustering performance. This, by no means, suggests that other methods do not hold merits. For example, methods relying on pairwise affinities (e.g., spectral clustering) have been very successful in many scenarios but still fail in some cases (e.g., overlapping clusters).