Neural Backward Filtering Forward Guiding

TL;DR

The paper introduces NBFFG, a neural framework for efficient inference in complex nonlinear stochastic processes on trees, combining a proxy linear process with neural residuals for improved accuracy.

Contribution

It presents a unified variational approach that leverages a proxy linear-Gaussian process and neural residuals, enabling unbiased pathwise sampling and better performance on complex tree-structured inference tasks.

Findings

Outperforms baselines on synthetic benchmarks.

Reduces training complexity from tree-size to path-length dependence.

Successfully applied to high-dimensional phylogenetic inference.

Abstract

Inference in nonlinear continuous stochastic processes on trees is challenging, particularly when observations are sparse and the topology is complex. Exact smoothing via Doob's $h$-transform is intractable for general nonlinear dynamics. We propose Neural Backward Filtering Forward Guiding (NBFFG), a unified framework for both discrete transitions and continuous diffusions. Our method constructs a variational posterior by leveraging a proxy linear-Gaussian process. This proxy process yields a closed-form backward filter that serves as a guide, steering the generative path toward high-likelihood regions. We then learn a neural residual to capture the non-linear discrepancies. This formulation allows for an unbiased pathwise subsampling scheme, reducing the training complexity from tree-size dependent to path-length dependent. Empirical results show that NBFFG outperforms baselines on synthetic benchmarks, and we demonstrate the method on a high-dimensional inference task in phylogenetic analysis with reconstruction of ancestral butterfly wing shapes.