A Faster Algorithm to Build New Users Similarity List in Neighbourhood-based Collaborative Filtering

TL;DR

This paper introduces the TwinSearch Algorithm, a faster method for updating user similarity lists in neighbourhood-based collaborative filtering, significantly reducing computational costs especially in cases with similar new users.

Contribution

The paper presents a novel algorithm that reduces the complexity of updating user similarity lists from O(n^2) to a much faster rate, improving scalability.

Findings

TwinSearch achieves 8x faster performance than traditional methods.

Theoretical analysis confirms reduced computational complexity.

Experimental results validate efficiency gains in real-world scenarios.

Abstract

Neighbourhood-based Collaborative Filtering (CF) has been applied in the industry for several decades, because of the easy implementation and high recommendation accuracy. As the core of neighbourhood-based CF, the task of dynamically maintaining users' similarity list is challenged by cold-start problem and scalability problem. Recently, several methods are presented on solving the two problems. However, these methods applied an $O(n^2)$ algorithm to compute the similarity list in a special case, where the new users, with enough recommendation data, have the same rating list. To address the problem of large computational cost caused by the special case, we design a faster ($O(\frac{1}{125}n^2)$) algorithm, TwinSearch Algorithm, to avoid computing and sorting the similarity list for the new users repeatedly to save the computational resources. Both theoretical and experimental results show that the TwinSearch Algorithm achieves better running time than the traditional method.