Local Popularity Based Collaborative Filters

TL;DR

This paper analyzes a local popularity-based collaborative filtering algorithm for binary random fields, showing that its effectiveness depends on cluster size and erasure rate, with zero error achievable above a certain threshold.

Contribution

It provides a theoretical analysis of a local popularity algorithm's performance in large matrices under high erasure rates, identifying a critical cluster size threshold.

Findings

BER approaches zero when cluster size exceeds threshold

Below threshold, BER is bounded away from zero

Performance depends on noise level and erasure rate

Abstract

Motivated by applications such as recommendation systems, we consider the estimation of a binary random field X obtained by row and column permutations of a block constant random matrix. The estimation of X is based on observations Y, which are obtained by passing entries of X through a binary symmetric channel (BSC) and an erasure channel. We focus on the analysis of a specific algorithm based on local popularity when the erasure rate approaches unity at a specified rate. We study the bit error rate (BER) in the limit as the matrix size approaches infinity. Our main result states that if the cluster size (that is, the size of the constancy blocks in the original matrix) is above a certain threshold, then the BER approaches zero, but below the threshold, the BER is lower bounded away from zero. The lower bound depends on the noise level in the observations and the size of the clusters in relation to the threshold. The threshold depends on the rate at which the erasure probability approaches unity.