# Bayesian inference of fitness landscapes via tree-structured branching processes

## TL;DR

This paper introduces FiTree, a new model that uses Bayesian inference to better understand how mutations and selection shape cancer evolution.

## Contribution

FiTree unifies probabilistic graphical models and population genetics to infer epistatic fitness landscapes from tumor mutation trees.

## Key findings

- FiTree outperforms existing methods in inferring fitness landscapes from simulated data.
- Application to a leukemia dataset reveals epistatic effects consistent with known biology.
- The model quantifies uncertainty in predicting future mutational events.

## Abstract

The complex dynamics of cancer evolution, driven by mutation and selection, underlies the molecular heterogeneity observed in tumors. The evolutionary histories of tumors of different patients can be encoded as mutation trees and reconstructed in high resolution from single-cell sequencing data, offering crucial insights for studying fitness effects of and epistasis among mutations. Existing models, however, either fail to separate mutation and selection or neglect the evolutionary histories encoded by the tumor phylogenetic trees.

We introduce FiTree, a tree-structured multi-type branching process model with epistatic fitness parameterization and a Bayesian inference scheme to learn fitness landscapes from single-cell tumor mutation trees. Through simulations, we demonstrate that FiTree outperforms state-of-the-art methods in inferring the fitness landscape underlying tumor evolution. Applying FiTree to a single-cell acute myeloid leukemia dataset, we identify epistatic fitness effects consistent with known biological findings and quantify uncertainty in predicting future mutational events. The new model unifies probabilistic graphical models of cancer progression with population genetics, offering a principled framework for understanding tumor evolution and informing therapeutic strategies.

The Python package FiTree and the analysis workflows are available at https://github.com/cbg-ethz/FiTree.