# SpaConTDS: A multimodal contrastive learning framework for identifying spatial domains by applying tuple disturbing strategy

## TL;DR

SpaConTDS is a new framework that uses multimodal contrastive learning and reinforcement learning to improve the identification of spatial domains in spatial transcriptomics data.

## Contribution

SpaConTDS introduces a novel tuple disturbing strategy and reinforcement learning for multimodal contrastive learning in spatial domain identification.

## Key findings

- SpaConTDS achieves state-of-the-art accuracy in spatial domain identification.
- The model outperforms existing methods in downstream tasks like denoising and UMAP visualization.
- SpaConTDS effectively integrates multiple tissue sections and corrects batch effects without alignment.

## Abstract

The rational utilization of multimodal spatial transcriptomics (ST) data enables accurate identification of spatial domains, which is essential for investigating cellular structure and functions. In this study, we proposed SpaConTDS, a novel framework that integrates reinforcement learning with self-supervised multimodal contrastive learning. SpaConTDS generates positive and negative samples through data augmentation and a pseudo-label tuple perturbation strategy, enabling the learning of fused representations that capture global semantics and cross-modal interactions. The model’s hyper-parameters are dynamically optimized using reinforcement learning. Extensive experiments across various resolutions and platforms demonstrate that SpaConTDS achieves state-of-the-art accuracy in spatial domain identification and outperforms existing methods in downstream tasks such as denoising, trajectory inference, and UMAP visualization. Moreover, SpaConTDS effectively integrates multiple tissue sections and corrects batch effects without requiring prior alignment. Compared to existing approaches, SpaConTDS offers more robust fused representations of multimodal data, providing researchers with a flexible and powerful tool for a wide range of spatial transcriptomics analyses.

Advancements in ST technologies have enabled researchers to simultaneously capture histological features, gene expression profiles, and spatial information. Unsupervised clustering of captured spots into spatial domains constitutes a fundamental component of spatial transcriptomics analysis, with the objective of delineating spatially coherent regions that are typically associated with distinct biological functions or tissue architectures. In this paper, we proposed a multi-modal spatial domain identification model called SpaConTDS. The innovations of SpaConTDS can be summarized as follows. Firstly, it utilizes a self-supervised multimodal contrastive learning method to effectively integrate gene expression and histopathological image information for spatial domain identification and alignment-free slice integration. Secondly, SpaConTDS uses reinforcement learning and global positive/negative sample construction strategies to adaptively capture fused representations that encompass interactions between modalities, which ensures that weak modalities are not neglected while avoiding the introduction of noise from image information. Thirdly, for multi-slice integration, negative samples and positive samples derived from the global similarity matrix can cover all slices, enabling SpaConTDS to automatically smooth the features of adjacent points both within and across slices without the need for slice alignment, thus learning more comprehensive cross-slice information and alleviating batch effects. The numerical results demonstrate that SpaConTDS exhibits superiority over existing methods in both spatial domain identification and integrated analysis on multiple slices. Moreover, the learned representations are applicable to various downstream tasks, including trajectory inference, gene expression denoising and uniform manifold approximation and projection (UMAP) visualization.