MatRec: Matrix Factorization for Highly Skewed Dataset

TL;DR

This paper introduces MatRec, a matrix factorization algorithm designed to effectively handle highly skewed datasets in recommender systems, improving computational efficiency and recommendation quality.

Contribution

The paper presents a novel matrix factorization approach that models data skewness explicitly, with interpretable formulas and competitive experimental results.

Findings

Achieves comparable results to state-of-the-art algorithms

Effectively models data skewness in recommender systems

Improves computational handling of skewed datasets

Abstract

Recommender systems is one of the most successful AI technologies applied in the internet cooperations. Popular internet products such as TikTok, Amazon, and YouTube have all integrated recommender systems as their core product feature. Although recommender systems have received great success, it is well known for highly skewed datasets, engineers and researchers need to adjust their methods to tackle the specific problem to yield good results. Inability to deal with highly skewed dataset usually generates hard computational problems for big data clusters and unsatisfactory results for customers. In this paper, we propose a new algorithm solving the problem in the framework of matrix factorization. We model the data skewness factors in the theoretic modeling of the approach with easy to interpret and easy to implement formulas. We prove in experiments our method generates comparably favorite results with popular recommender system algorithms such as Learning to Rank , Alternating Least Squares and Deep Matrix Factorization.