Efficiently Maintaining Next Basket Recommendations under Additions and Deletions of Baskets and Items

TL;DR

This paper introduces efficient algorithms for updating next basket recommendation models in response to additions and deletions of user baskets and items, ensuring privacy compliance and real-time responsiveness.

Contribution

It proposes novel incremental and decremental update algorithms for next basket recommendation models, enabling fast model adjustments under GDPR data deletion requirements.

Findings

Constant update time for incremental changes (~0.2 ms)

Linear update time for deletions (<1 ms)

Effective implementation in Spark Structured Streaming

Abstract

Recommender systems play an important role in helping people find information and make decisions in today's increasingly digitalized societies. However, the wide adoption of such machine learning applications also causes concerns in terms of data privacy. These concerns are addressed by the recent "General Data Protection Regulation" (GDPR) in Europe, which requires companies to delete personal user data upon request when users enforce their "right to be forgotten". Many researchers argue that this deletion obligation does not only apply to the data stored in primary data stores such as relational databases but also requires an update of machine learning models whose training set included the personal data to delete. We explore this direction in the context of a sequential recommendation task called Next Basket Recommendation (NBR), where the goal is to recommend a set of items based on a user's purchase history. We design efficient algorithms for incrementally and decrementally updating a state-of-the-art next basket recommendation model in response to additions and deletions of user baskets and items. Furthermore, we discuss an efficient, data-parallel implementation of our method in the Spark Structured Streaming system. We evaluate our implementation on a variety of real-world datasets, where we investigate the impact of our update techniques on several ranking metrics and measure the time to perform model updates. Our results show that our method provides constant update time efficiency with respect to an additional user basket in the incremental case, and linear efficiency in the decremental case where we delete existing baskets. With modest computational resources, we are able to update models with a latency of around 0.2~milliseconds regardless of the history size in the incremental case, and less than one millisecond in the decremental case.