Selecting the rank of truncated SVD by Maximum Approximation Capacity

TL;DR

This paper introduces an information-theoretic method based on maximum approximation capacity to select the optimal rank in truncated SVD, providing a principled approach that competes with existing techniques.

Contribution

It formulates the rank selection as a noisy channel coding problem, offering a novel, theoretically grounded criterion for model order determination in SVD.

Findings

The method effectively determines the optimal rank in simulations.

It competes favorably with state-of-the-art model selection techniques.

The approach provides a principled, information-theoretic criterion for rank selection.

Abstract

Truncated Singular Value Decomposition (SVD) calculates the closest rank-$k$ approximation of a given input matrix. Selecting the appropriate rank $k$ defines a critical model order choice in most applications of SVD. To obtain a principled cut-off criterion for the spectrum, we convert the underlying optimization problem into a noisy channel coding problem. The optimal approximation capacity of this channel controls the appropriate strength of regularization to suppress noise. In simulation experiments, this information theoretic method to determine the optimal rank competes with state-of-the art model selection techniques.