Learning with the Weighted Trace-norm under Arbitrary Sampling Distributions

TL;DR

This paper analyzes the weighted trace-norm for learning from arbitrary sampling distributions, introduces a corrected variant with strong guarantees, and demonstrates its practical effectiveness over the standard approach.

Contribution

It identifies limitations of the standard weighted trace-norm under non-product distributions and proposes a corrected variant with proven learning guarantees and improved practical performance.

Findings

The standard weighted trace-norm can fail with non-product sampling distributions.

A corrected weighted trace-norm variant provides strong theoretical guarantees.

Empirical results show the corrected variant outperforms the standard method.

Abstract

We provide rigorous guarantees on learning with the weighted trace-norm under arbitrary sampling distributions. We show that the standard weighted trace-norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), present a corrected variant for which we establish strong learning guarantees, and demonstrate that it works better in practice. We provide guarantees when weighting by either the true or empirical sampling distribution, and suggest that even if the true distribution is known (or is uniform), weighting by the empirical distribution may be beneficial.