Matrix completion with data-dependent missingness probabilities

TL;DR

This paper introduces new methods for matrix completion where missingness depends on the entry values, providing estimators that are consistent without tuning parameters and a way to estimate the missingness function.

Contribution

It proposes two novel estimators for matrix completion with data-dependent missingness probabilities and a method to estimate the unknown missingness function.

Findings

Estimators are consistent under a low rank assumption.

No tuning parameters are required for the proposed estimators.

A consistent estimator for the missingness probability function is developed.

Abstract

The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$.