IFA: Interaction Fidelity Attention for Entire Lifelong Behaviour Sequence Modeling

TL;DR

The paper introduces IFA, an efficient attention-based model for lifelong user behavior sequence modeling that reduces information loss and computational cost, improving recommendation accuracy in online systems.

Contribution

The paper proposes IFA, a novel method that models full lifelong sequences with linear transformer and target item relationships, addressing efficiency and information loss issues.

Findings

Significant improvement in recommendation accuracy.

Reduced computational latency in online systems.

Effective modeling of full user behavior sequences.

Abstract

The lifelong user behavior sequence provides abundant information of user preference and gains impressive improvement in the recommendation task, however increases computational consumption significantly. To meet the severe latency requirement in online service, a short sub-sequence is sampled based on similarity to the target item. Unfortunately, items not in the sub-sequence are abandoned, leading to serious information loss. In this paper, we propose a new efficient paradigm to model the full lifelong sequence, which is named as \textbf{I}nteraction \textbf{F}idelity \textbf{A}ttention (\textbf{IFA}). In IFA, we input all target items in the candidate set into the model at once, and leverage linear transformer to reduce the time complexity of the cross attention between the candidate set and the sequence without any interaction information loss. We also additionally model the relationship of all target items for optimal set generation, and design loss function for better consistency of training and inference. We demonstrate the effectiveness and efficiency of our model by off-line and online experiments in the recommender system of Kuaishou.