Learning Deep Tree-based Retriever for Efficient Recommendation: Theory and Method

TL;DR

This paper introduces Deep Tree-based Retriever (DTR), a novel method that improves recommendation efficiency by explicitly modeling tree structures with a multi-class classification approach, outperforming previous binary classification strategies.

Contribution

The paper proposes a multi-class classification framework for tree-based recommendation, along with loss rectification and sampling techniques, enhancing efficiency and alignment with the max-heap assumption.

Findings

DTR achieves superior recommendation accuracy on real-world datasets.

The proposed methods improve generalization and efficiency of tree-based recommendation models.

Theoretical analysis confirms DTR's strong generalization capabilities.

Abstract

Although advancements in deep learning have significantly enhanced the recommendation accuracy of deep recommendation models, these methods still suffer from low recommendation efficiency. Recently proposed tree-based deep recommendation models alleviate the problem by directly learning tree structure and representations under the guidance of recommendation objectives. To guarantee the effectiveness of beam search for recommendation accuracy, these models strive to ensure that the tree adheres to the max-heap assumption, where a parent node's preference should be the maximum among its children's preferences. However, they employ a one-versus-all strategy, framing the training task as a series of independent binary classification objectives for each node, which limits their ability to fully satisfy the max-heap assumption. To this end, we propose a Deep Tree-based Retriever (DTR for short) for efficient recommendation. DTR frames the training task as a softmax-based multi-class classification over tree nodes at the same level, enabling explicit horizontal competition and more discriminative top-k selection among them, which mimics the beam search behavior during training. To mitigate the suboptimality induced by the labeling of non-leaf nodes, we propose a rectification method for the loss function, which further aligns with the max-heap assumption in expectation. As the number of tree nodes grows exponentially with the levels, we employ sampled softmax to approximate optimization and thereby enhance efficiency. Furthermore, we propose a tree-based sampling method to reduce the bias inherent in sampled softmax. Theoretical results reveal DTR's generalization capability, and both the rectification method and tree-based sampling contribute to improved generalization. The experiments are conducted on four real-world datasets, validating the effectiveness of the proposed method.