Bayesian Singular Value Regularization via a Cumulative Shrinkage Process

TL;DR

This paper introduces a hierarchical Bayesian prior with a cumulative shrinkage process for low-rank matrix estimation, effectively shrinking insignificant singular values to zero and improving inference accuracy.

Contribution

It develops a novel prior combining spike and slab components with a cumulative shrinkage process for Bayesian low-rank matrix inference.

Findings

Competitive performance with existing priors

Effective shrinkage of redundant singular values

Minimal additional computational cost

Abstract

This study proposes a novel hierarchical prior for inferring possibly low-rank matrices measured with noise. We consider three-component matrix factorization, as in singular value decomposition, and its fully Bayesian inference. The proposed prior is specified by a scale mixture of exponential distributions that has spike and slab components. The weights for the spike/slab parts are inferred using a special prior based on a cumulative shrinkage process. The proposed prior is designed to increasingly aggressively push less important, or essentially redundant, singular values toward zero, leading to more accurate estimates of low-rank matrices. To ensure the parameter identification, we simulate posterior draws from an approximated posterior, in which the constraints are slightly relaxed, using a No-U-Turn sampler. By means of a set of simulation studies, we show that our proposal is competitive with alternative prior specifications and that it does not incur significant additional computational burden. We apply the proposed approach to sectoral industrial production in the United States to analyze the structural change during the Great Moderation period.