On Application of Block Kaczmarz Methods in Matrix Factorization

TL;DR

This paper introduces a block Kaczmarz method for matrix factorization that significantly reduces computation time and memory usage while maintaining comparable accuracy, making it suitable for large-scale data applications.

Contribution

The paper proposes and empirically tests a block Kaczmarz solver as an efficient alternative to traditional least-squares methods in matrix factorization.

Findings

Block Kaczmarz achieves faster factorization with similar accuracy.

Optimal block sizes reduce runtime and memory usage.

Method is effective for large-scale data matrices.

Abstract

Matrix factorization techniques compute low-rank product approximations of high dimensional data matrices and as a result, are often employed in recommender systems and collaborative filtering applications. However, many algorithms for this task utilize an exact least-squares solver whose computation is time consuming and memory-expensive. In this paper we discuss and test a block Kaczmarz solver that replaces the least-squares subroutine in the common alternating scheme for matrix factorization. This variant trades a small increase in factorization error for significantly faster algorithmic performance. In doing so we find block sizes that produce a solution comparable to that of the least-squares solver for only a fraction of the runtime and working memory requirement.