Dual Advancement of Representation Learning and Clustering for Sparse and Noisy Images

TL;DR

DARLC is a novel framework that combines contrastive learning and clustering with graph attention networks to improve representation quality and clustering accuracy for sparse, noisy images like spatial gene expression data.

Contribution

It introduces an integrated end-to-end approach that enhances representations and clustering for SNIs, addressing class collision and noise issues with innovative techniques.

Findings

Outperforms state-of-the-art in image clustering

Improves representation quality for noisy images

Effective across 12 diverse datasets

Abstract

Sparse and noisy images (SNIs), like those in spatial gene expression data, pose significant challenges for effective representation learning and clustering, which are essential for thorough data analysis and interpretation. In response to these challenges, we propose Dual Advancement of Representation Learning and Clustering (DARLC), an innovative framework that leverages contrastive learning to enhance the representations derived from masked image modeling. Simultaneously, DARLC integrates cluster assignments in a cohesive, end-to-end approach. This integrated clustering strategy addresses the "class collision problem" inherent in contrastive learning, thus improving the quality of the resulting representations. To generate more plausible positive views for contrastive learning, we employ a graph attention network-based technique that produces denoised images as augmented data. As such, our framework offers a comprehensive approach that improves the learning of representations by enhancing their local perceptibility, distinctiveness, and the understanding of relational semantics. Furthermore, we utilize a Student's t mixture model to achieve more robust and adaptable clustering of SNIs. Extensive experiments, conducted across 12 different types of datasets consisting of SNIs, demonstrate that DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions. Code is available at https://github.com/zipging/DARLC.