Diffusion State Distances: Multitemporal Analysis, Fast Algorithms, and Applications to Biological Networks

TL;DR

This paper introduces diffusion state distance (DSD), a data-dependent metric that captures multiscale structures in high-dimensional data through a diffusion process, enabling efficient analysis of biological networks.

Contribution

It develops a theoretical framework for DSD based on multitemporal diffusion equilibria and proposes new algorithms for denoising and dimension reduction.

Findings

DSD effectively captures multiscale data structures.

Algorithms show improved speed and accuracy on biological networks.

Theoretical analysis supports the robustness of DSD.

Abstract

Data-dependent metrics are powerful tools for learning the underlying structure of high-dimensional data. This article develops and analyzes a data-dependent metric known as diffusion state distance (DSD), which compares points using a data-driven diffusion process. Unlike related diffusion methods, DSDs incorporate information across time scales, which allows for the intrinsic data structure to be inferred in a parameter-free manner. This article develops a theory for DSD based on the multitemporal emergence of mesoscopic equilibria in the underlying diffusion process. New algorithms for denoising and dimension reduction with DSD are also proposed and analyzed. These approaches are based on a weighted spectral decomposition of the underlying diffusion process, and experiments on synthetic datasets and real biological networks illustrate the efficacy of the proposed algorithms in terms of both speed and accuracy. Throughout, comparisons with related methods are made, in order to illustrate the distinct advantages of DSD for datasets exhibiting multiscale structure.