# Mathematical strategies for predicting resistant subpopulations from scRNAseq data of a PANC-1 3D tissue model: Insight into gemcitabine resistance and TGFB1-induced invasion and EMT

## TL;DR

This study uses 3D cell models and machine learning to identify genes linked to drug resistance and cancer cell invasion in pancreatic cancer.

## Contribution

A novel mathematical approach and machine learning framework (gSELECT) to predict resistant subpopulations from scRNA-seq data.

## Key findings

- TGFB1 induces an invasive phenotype and drug resistance marked by EMT-related gene changes.
- Machine learning predicted a GEM-sensitive subpopulation consistent with experimental survival data.
- Mutual information genes distinguishing resistant and sensitive cells were identified and validated.

## Abstract

Pancreatic ductal adenocarcinoma is characterized by high levels of chemoresistance and aggressive progression of the disease, which is a major challenge for effective treatment. Conventional 2D cultures capture therapy resistance only to a limited extent, whereas 3D cultures may better reflect relevant conditions.

We established 3D PANC-1 tissue models based on a decellularized porcine jejunum with niche-specific drug response to gemcitabine (GEM) treatment. TGFB1 induced invasion and further drug resistance. Thus, we performed scRNA-seq after treatment with GEM, TGFB1 stimulation, or both. Data were analyzed using standard approaches and a novel mutual-information-based machine learning framework (gSELECT). Candidate genes were further evaluated through enrichment and survival analyses.

Additionally, we present a novel mathematical approach as proof of concept for analyzing differences in gene expression between groups seemingly similar with respect to a projection such as t-SNE or UMAP (e.g., GEM-treated and untreated cells). For this, we stratified control cells by similarity to GEM-treated survivors, yielding predicted-resistant and predicted-sensitive subgroups for downstream analysis.

Pre-analysis using machine learning and comparative analyses of single-cell RNA sequencing data showed only minor differences in gene expression in response to GEM treatment, whereas TGFB1 induced an invasive phenotype characterized by EMT-related transcriptional changes, including downregulation of cytokeratins.

Laboratory experiments showed that ∼75 % of PANC-1 cells survived GEM in 3D, indicating intrinsic resistance. Our mathematical approach using machine learning predicted a GEM-sensitive subpopulation consistent with these findings. Comparative analyses revealed mutual information (MI) genes distinguishing sensitive from resistant cells, several of which were supported by literature and survival data.

Computational analysis of scRNA-seq data from 3D-cultured PANC-1 cells provides a useful framework for studying treatment effects. The potential relevance of the identified MI genes is supported by further in silico analyses. Based on in silico analyses, we demonstrate the analytical value of our mathematical approach and identify candidate genes for further functional and therapeutic validation.