Comparative Study of Inference Methods for Interpolative Decomposition

TL;DR

This paper introduces a Bayesian probabilistic model with ARD for interpolative decomposition, improving low-rank approximation and feature selection by automatically determining relevant factors, and demonstrates superior performance on real datasets.

Contribution

The paper presents a novel Bayesian ID method with ARD that automatically determines the latent dimension, enhancing reconstruction accuracy over fixed-rank approaches.

Findings

Bayesian ID with ARD achieves lower reconstruction errors.

The model effectively identifies relevant features in real datasets.

Automatic relevance determination improves low-rank approximation.

Abstract

In this paper, we propose a probabilistic model with automatic relevance determination (ARD) for learning interpolative decomposition (ID), which is commonly used for low-rank approximation, feature selection, and identifying hidden patterns in data, where the matrix factors are latent variables associated with each data dimension. Prior densities with support on the specified subspace are used to address the constraint for the magnitude of the factored component of the observed matrix. Bayesian inference procedure based on Gibbs sampling is employed. We evaluate the model on a variety of real-world datasets including CCLE $EC50$, CCLE $IC50$, Gene Body Methylation, and Promoter Methylation datasets with different sizes, and dimensions, and show that the proposed Bayesian ID algorithms with automatic relevance determination lead to smaller reconstructive errors even compared to vanilla Bayesian ID algorithms with fixed latent dimension set to matrix rank.