A Note on Element-wise Matrix Sparsification via a Matrix-valued Bernstein Inequality

TL;DR

This paper introduces a straightforward element-wise matrix sparsification method that zeroes small elements and probabilistically retains larger ones, with analysis based on a recent matrix Bernstein inequality, improving understanding of approximation accuracy.

Contribution

It proposes a new simple element-wise sparsification algorithm and provides theoretical analysis using a modern matrix Bernstein inequality, comparing favorably with existing methods.

Findings

The algorithm effectively sparsifies matrices while maintaining approximation quality.

Theoretical bounds on approximation error are established using a non-commutative Bernstein inequality.

Compared bounds outperform or match existing sparsification techniques.

Abstract

Given an n x n matrix A, we present a simple, element-wise sparsification algorithm that zeroes out all sufficiently small elements of A and then retains some of the remaining elements with probabilities proportional to the square of their magnitudes. We analyze the approximation accuracy of the proposed algorithm using a recent, elegant non-commutative Bernstein inequality, and compare our bounds with all existing (to the best of our knowledge) element-wise matrix sparsification algorithms.