Sparse Bayesian multidimensional scaling(s)

TL;DR

This paper introduces sparse variants of Bayesian multidimensional scaling (BMDS) that significantly reduce computational complexity, enabling scalable analysis of large dissimilarity datasets while maintaining accuracy, with applications in phylogeography and document clustering.

Contribution

The authors develop and compare two sparse BMDS methods, L-sBMDS and B-sBMDS, that apply subset-based likelihood calculations to improve scalability and efficiency.

Findings

Achieve up to 40-fold speedup with negligible accuracy loss

Prove posterior consistency under simplified conditions

Demonstrate practical applications in influenza phylogeography and manuscript clustering

Abstract

Bayesian multidimensional scaling (BMDS) is a probabilistic dimension reduction tool that allows one to model and visualize data consisting of dissimilarities between pairs of objects. Although BMDS has proven useful within, e.g., Bayesian phylogenetic inference, its likelihood and gradient calculations require a burdensome order of $N^2$ floating-point operations, where $N$ is the number of data points. Thus, BMDS becomes impractical as $N$ grows large. We propose and compare two sparse versions of BMDS (sBMDS) that apply log-likelihood and gradient computations to subsets of the observed dissimilarity matrix data. Landmark sBMDS (L-sBMDS) extracts columns, while banded sBMDS (B-sBMDS) extracts diagonals of the data. These sparse variants let one specify a time complexity between $N^2$ and $N$. Under simplified settings, we prove posterior consistency for subsampled distance matrices. Through simulations, we examine the accuracy and computational efficiency across all models using both the Metropolis-Hastings and Hamiltonian Monte Carlo algorithms. We observe approximately 3-fold, 10-fold and 40-fold speedups with negligible loss of accuracy, when applying the sBMDS likelihoods and gradients to 500, 1,000 and 5,000 data points with 50 bands (landmarks); these speedups only increase with the size of data considered. Finally, we apply the sBMDS variants to: 1) the phylogeographic modeling of multiple influenza subtypes to better understand how these strains spread through global air transportation networks and 2) the clustering of ArXiv manuscripts based on low-dimensional representations of article abstracts. In the first application, sBMDS contributes to holistic uncertainty quantification within a larger Bayesian hierarchical model. In the second, sBMDS provides uncertainty quantification for a downstream modeling task.