Deviance Matrix Factorization

TL;DR

This paper introduces a flexible matrix factorization method based on deviance loss, extending SVD to generalized linear models, with theoretical support, practical algorithms, and diverse applications.

Contribution

It develops a novel deviance-based matrix factorization approach leveraging GLM theory, supporting structural zeros, and providing tools for model selection and validation.

Findings

Method outperforms traditional SVD in robustness and flexibility.

Supports structural zeros via entry weights.

Validated through simulations and diverse real-world datasets.

Abstract

We investigate a general matrix factorization for deviance-based data losses, extending the ubiquitous singular value decomposition beyond squared error loss. While similar approaches have been explored before, our method leverages classical statistical methodology from generalized linear models (GLMs) and provides an efficient algorithm that is flexible enough to allow for structural zeros via entry weights. Moreover, by adapting results from GLM theory, we provide support for these decompositions by (i) showing strong consistency under the GLM setup, (ii) checking the adequacy of a chosen exponential family via a generalized Hosmer-Lemeshow test, and (iii) determining the rank of the decomposition via a maximum eigenvalue gap method. To further support our findings, we conduct simulation studies to assess robustness to decomposition assumptions and extensive case studies using benchmark datasets from image face recognition, natural language processing, network analysis, and biomedical studies. Our theoretical and empirical results indicate that the proposed decomposition is more flexible, general, and robust, and can thus provide improved performance when compared to similar methods. To facilitate applications, an R package with efficient model fitting and family and rank determination is also provided.