Combining Microscopy Data and Metadata for Reconstruction of Cellular Traction Forces Using a Hybrid Vision Transformer-U-Net

TL;DR

This paper introduces ViT+UNet, a hybrid deep learning model combining Vision Transformer and U-Net, which improves traction force prediction accuracy, generalizes across scales and noise, and incorporates cell-type metadata for better inference in traction force microscopy.

Contribution

The study presents a novel hybrid architecture that outperforms existing models in TFM analysis and effectively integrates metadata for enhanced prediction.

Findings

Outperforms standalone U-Net and Vision Transformer models.

Generalizes well across different spatial scales and noise levels.

Incorporates cell-type metadata to improve prediction accuracy.

Abstract

Traction force microscopy (TFM) is a widely used technique for quantifying the forces that cells exert on their surrounding extracellular matrix. Although deep learning methods have recently been applied to TFM data analysis, several challenges remain-particularly achieving reliable inference across multiple spatial scales and integrating additional contextual information such as cell type to improve accuracy. In this study, we propose ViT+UNet, a robust deep learning architecture that integrates a U-Net with a Vision Transformer. Our results demonstrate that this hybrid model outperforms both standalone U-Net and Vision Transformer architectures in predicting traction force fields. Furthermore, ViT+UNet exhibits superior generalization across diverse spatial scales and varying noise levels, enabling its application to TFM datasets obtained from different experimental setups and imaging systems. By appropriately structuring the input data, our approach also allows the inclusion of metadata, in our case cell-type information, to enhance prediction specificity and accuracy.