Slate-Aware Ranking for Recommendation

TL;DR

This paper introduces Slate-Aware Ranking (SAR), a novel method that improves candidate set quality in slate recommender systems by implicitly modeling item relations during the ranking stage, enhancing overall relevance and diversity.

Contribution

The paper proposes a new slate-aware ranking approach that enhances candidate set quality by modeling item relations during ranking, improving subsequent re-ranking effectiveness.

Findings

SAR improves recommendation relevance and diversity.

Experimental results show SAR outperforms baseline methods.

Online A/B tests confirm SAR's effectiveness in real systems.

Abstract

We see widespread adoption of slate recommender systems, where an ordered item list is fed to the user based on the user interests and items' content. For each recommendation, the user can select one or several items from the list for further interaction. In this setting, the significant impact on user behaviors from the mutual influence among the items is well understood. The existing methods add another step of slate re-ranking after the ranking stage of recommender systems, which considers the mutual influence among recommended items to re-rank and generate the recommendation results so as to maximize the expected overall utility. However, to model the complex interaction of multiple recommended items, the re-ranking stage usually can just handle dozens of candidates because of the constraint of limited hardware resource and system latency. Therefore, the ranking stage is still essential for most applications to provide high-quality candidate set for the re-ranking stage. In this paper, we propose a solution named Slate-Aware ranking (SAR) for the ranking stage. By implicitly considering the relations among the slate items, it significantly enhances the quality of the re-ranking stage's candidate set and boosts the relevance and diversity of the overall recommender systems. Both experiments with the public datasets and internal online A/B testing are conducted to verify its effectiveness.

Tables

Table 1. Table 1. Notations and Descriptions

Notation	Description
$u$	User request with user profile and context
$i$	An recommendation item
$C$	The candidate set
$N$	The size of the candidate set
$M$	The number of prediction tasks
$S$	The recommended slate chosen from $C$
$K$	The size of the recommended slate
$i_k$	The $k$th item of $S$
$R(S,u)$	The reward for recommending $S$ given $u$
$r(i,u)$	The reward for recommending the item $i$ given $u$
$o_k$	Observation probability of the $k$th item in the slate
$e_u$	Concatenated embeddings for the user features
$e_i$	Concatenated embeddings for the item features
$e_s$	Concatenated embeddings for the slate-wise features
$l_u$	Encoding results of the user encoder network
$l_s$	Encoding results of the slate encoder network
$\lambda$	The weight for the similarity loss
${\omega }_m$	The weight for the loss of the $m$th task

Table 2. Table 2. Offline Performance for Industrial Dataset

$Model$	$AU{C}_{ctr}$	$RelGain$	$MA{E}_{wt}$	$RelGain$
$Baseline$	0.7841	0$\%$	11.3858	0$\%$
$PF{D}_{student}$(Xu et al., 2020)	0.7849	+0.10$\%$	11.3760	+0.09$\%$
$SA{R}_{SumPool}$	0.7871	+0.40$\%$	11.3672	+0.16$\%$
$SA{R}_{LSTM}$	0.7872	+0.41$\%$	11.3720	+0.12$\%$
$SA{R}_{Attn}$	0.7875	+0.43$\%$	11.3631	+0.20$\%$

Table 3. Table 3. Offline Performance for MovieLens-1M

Backbone	$Baseline$	$PF{D}_{student}$		$SA{R}_{Attn}$
	$AUC$	$AUC$	$RelGain$	$AUC$	$RelGain$
FM(Rendle, 2010)	0.8065	0.8062	-0.03%	0.8079	+0.17%
Wide&Deep(Cheng et al., 2016)	0.8055	0.8063	+0.10%	0.8087	+0.40%
NCF(He et al., 2017)	0.8062	0.8086	+0.30%	0.8110	+0.60%

Table 4. Table 4. Online Performance

	App Stay Time per Person	Page View per Person
$Gain$	0.745$\%$	0.9$\%$

Table 5. Table 5. The Gini Index of Online Traffic

$Model$	$ItemID$	$RelGain$	$ItemCat$	$RelGain$
$Baseline$	0.8829	0$\%$	0.7253	0$\%$
$SA{R}_{Attn}$	0.8818	+0.12$\%$	0.7126	+1.75$\%$

Equations

$$\underset{S}{\max }R(S,u;\theta ),s.t.S\subseteq C$$

$$R(S,u)=\sum _{k=1}^K{o}_{k}r(i_k,u),s.t.S=\{i_1,i_2,...,i_K\}$$

$$p_1,p_2,...,p_M=f_{base}(i,u)$$

$$r(i,u)=\Phi (p_1,p_2,...,p_M)$$

$$p_1,p_2,...,p_M=f_{ideal}(i,u,S)$$

$$e_u=Embeddin{g}_u(u)$$

$$e_i=Embeddin{g}_i(i)$$

$$p_1,p_2,...,p_M=f_{base}(i,u)=Backbone(e_u,e_i)$$

$$e_s=Embeddin{g}_s(S)$$

$$l_s=Encode{r}_s(e_s,e_u)$$

$$l_u=Encode{r}_u(e_u)$$

$$d=Decoder(l,e_u)$$

$$p_1,p_2,...,p_M=Backbone(e_u,e_i,d)$$

$$Loss(X,Y_{1:M})=\sum _{m=1}^M{\omega }_m{L}_m(X,Y_m)+\lambda Los{s}_{sim}$$

Full text

Slate Recommendation; Ranking; Re-Ranking; Recommender Systems; Multi-Task Learning; Privileged Information; Distillation

††journalyear: 2023††copyright: acmcopyright††conference: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining; February 27-March 3, 2023; Singapore, Singapore††booktitle: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (WSDM ’23), February 27-March 3, 2023, Singapore, Singapore††price: 15.00††doi: 10.1145/3539597.3570380††isbn: 978-1-4503-9407-9/23/02††ccs: Information systems Recommender systems

1. Introduction

In this era of information explosion, recommender systems are useful tools to filter out valuable information for the users and already impact almost every facet of people’s lives. Typically, for each user request, recommender systems select a slate of items for presentation from the large candidate set based on its understanding on the user’s intention. There are two main reasons for returning a slate of items rather than a single item. First, in many recommendation scenarios, the user can observe multiple items in the screen at once. Moreover, the overload of the recommender systems is greatly alleviated by lowering the QPS (Query-Per-Second) of interactions between the users and system servers. After receiving the recommendation slate, the user can scroll the screen and consume the items relevant to his current interest. A new user request will be issued once the slate of items are all presented. Please refer to the ”watch next recommendation” scenario on Youtube displayed at the right pane of Figure 1 for a representative slate recommendation scenario.

Generally speaking, typical recommender systems take the multi-stage design and consist of at least two steps (Covington et al., 2016), namely matching and ranking. For some later systems (Wang et al., 2020), additional steps, including matching, pre-ranking, ranking and re-ranking, are introduced for better trade-off between recommendation performance and system latency. From matching to re-ranking, more and more powerful models and comprehensive features are utilized yet with a decreasing candidate size. The steps of matching and pre-ranking usually utilize highly efficient methods to filter out relevant items from a vast candidate pool and care coverage more than precision. What’s more, with limited hardware resource and system latency, the re-ranking stage normally just handles dozens of candidates by design. Hence, the ranking stage, which selects top tens of items from a pool of hundreds to thousands with high precision, plays an essential role for the recommendation performance.

Traditionally, the models in the ranking stage compute a uni-variate score for each given user-item pair and construct the slate based on the order of item scores (Cremonesi et al., 2010). As the top ranked items take more chance to be observed or preferred, the overall efficiency should be optimized in this way if we assume the user’s preference for a item is stable. Nevertheless, it is well known that the user interaction and feedback depends on not only the corresponding item’s quality and relevance but also the contextual items’ impact (Bello et al., 2018; Pang et al., 2020; Deng et al., 2018; Zhuang et al., 2018; Pei et al., 2019; Wang et al., 2019; Jiang et al., 2018; Zhao et al., 2017; Gong et al., 2019; Wei et al., 2020; Feng et al., 2021b; Ai et al., 2018a, 2019; Liu et al., 2021; Feng et al., 2021a). As a result, the traditional models in the ranking stage, which just accept a given user-item pair’s features to learn a uni-variate scoring function, is incapable of taking into account the complex influence exerted by the surrounding items. And the recommendation performance tends to be sub-optimal.

To tackle this challenge, various algorithms have been devised and implemented in the re-ranking stage (Bello et al., 2018; Pang et al., 2020; Zhuang et al., 2018; Pei et al., 2019; Wang et al., 2019; Zhao et al., 2017; Gong et al., 2019; Wei et al., 2020; Feng et al., 2021b; Ai et al., 2018a, 2019; Feng et al., 2021a). Acting as the last step after the matching and ranking stage, the re-ranking stage obtains the top ranking items from the ranking stage as candidates and refines into the final recommendation results. Admittedly, these re-ranking methods achieved success in many industrial applications. Nevertheless, to account for the complex relations among the displayed items, these models are usually much more complex than the models in the ranking stage. Considering the limited system latency, the re-ranking models usually can only handle dozens of items ranked top by the ranking stage. Thus, it is essential for the ranking stage to provide highly relevant yet diverse top item set. Nonetheless, as the ranking stage is oblivious to the overall recommendation list and does not consider the cross item impact, the re-ranking stage are likely to receive unsatisfactory candidate set, thereby negatively affecting the overall performance of recommender systems.

There are some challenges for addressing the issue of cross item impact in the ranking stage. In contrast to the re-ranking stage, the ranking stage just provides competitive candidates to the next stage and cannot determine the final recommendation results. Therefore, we cannot use the slate-wise features in the ranking stage for model training. Moreover, with much larger candidate set, it is also infeasible to deploy these sophisticated and time-consuming models used in the re-ranking stage, such as the sequential generation algorithms (Bello et al., 2018; Zhuang et al., 2018; Gong et al., 2019), to the ranking stage.

To model the mutual influence among the items of a recommendation slate in the ranking stage, one viable solution is to borrow the ideas from the works of privileged features, which are defined as the features only available for training, and distillation (Lopez-Paz et al., 2015; Xu et al., 2020). Besides the base ranking model, a teacher model utilizing both the base ranking features and the slate-wise features can be trained. With knowledge distillation, valuable information can be transferred from the teacher model to the base ranking model served online. But this approach need roughly double the training resources. In addition, without specialized attention to the privileged features, the distillation methods may generate sub-optimal performance, which is verified by our experiment.

We propose the method of Slate-Aware Ranking (SAR), which utilizes the whole recommendation slate’s information during training, for the ranking stage. Specifically, with an encoder network, we construct the slate-wise features based on the whole item sequence in a slate and encode them into the latent space, which is then processed by a decoder network and concatenated to the input of upper ranking modules so as to help to minimize the training loss. Meanwhile, with another encoder network, we also encode the user features to an embedding vector that matches the slate’s embedding well. The rationality is that the recommender systems are inclined to recommend results with homogeneous patterns based on the user’s previous behaviors, which can be memorized by the user encoder network. As we cannot figure out the actual recommendation slate during prediction, we use the embedding vector from user features instead. The most similar works come from (Liu et al., 2021; Jiang et al., 2018), which also encode the information of the whole slate to the latent space. But as a generation model, they can just use the vector-product based DNN model with limited expressive capacity. Previous works (Wang et al., 2020; Zhu et al., 2018; Xu et al., 2020) have shown the obviously superior performance of incorporating complex deep models over vector-product form networks. Moreover, the generative methods show stochastic behavior because of the Reconstruction-Concentration dilemma (Liu et al., 2021). Finally, they need input the best reward, which is hard to define for multi-task applications, as a generation condition during inference.

We summarize our main contributions below.

• •

We design an effective solution to implicitly model the mutual influence on user behaviors among the slate items in the ranking stage, which generates significantly better recommendation relevance and diversity by boosting the quality of the re-ranking stage’s candidate set.

• •

Compared with the distillation and privileged feature methods (Lopez-Paz et al., 2015; Xu et al., 2020), we achieve significant performance gains with around half training resources.

• •

We conduct extensive offline and online experiments to evaluate and understand the effectiveness of our method. Online A/B testing in one of the world’s largest content recommendation platforms shows significant improvement of the business metrics.

2. Related Work

2.1. Re-ranking Models for Slate Recommendation

To model the mutual influence among slate items, various algorithms have been devised and implemented in the re-ranking stage. Acting as the last step after the matching and ranking stage, the re-ranking stage obtains the top ranking items from the ranking stage as candidates and refines into the final recommendation results.

These re-ranking methods can be roughly classified into three categories. The first category comprises the methods of one-step generator, which extract both item-wise features and slate-wise features for each item, predict its relevance score and perform greedy ranking. Though DLCM (Ai et al., 2018a), PRM (Pei et al., 2019), GSF (Ai et al., 2019) and SetRank (Pang et al., 2020) leverage different modelling methods, they all belong to this type. Admittedly, for some methods (Ai et al., 2018a), the scoring function may vary based on the permutation of the initial ranking list, all of them (Ai et al., 2018a; Pei et al., 2019; Ai et al., 2019; Pang et al., 2020) are not sensitive to the order of the final recommendation slate. Second, for the methods of sequential generator (Bello et al., 2018; Zhuang et al., 2018; Gong et al., 2019), besides the slate-wise feature extraction and encoding, these methods explicitly model the sequential impact on the current item from the previously chosen ones with uni-directional sequential models, such as LSTM networks (Greff et al., 2016). Nonetheless, as a sequential generation model, they have to ignore the current item’s influence on the previous ones, which does not adhere to the true distributions. Finally, for the methods of generator and evaluator (Wang et al., 2019; Feng et al., 2021a; Wei et al., 2020; Feng et al., 2021b), they initially generate multiple slate candidates with different methods mentioned above and use a separate evaluator model to compute the expected utility of each slate. As the evaluator is able to utilize the bi-directional models, such as bi-directional LSTM and transformer networks (Vaswani et al., 2017), they are able to capture both the forward and backward impact among the recommended items so as to ensure better recommendation quality.

Please note that there are explicit reasons why we cannot directly apply these algorithms to the ranking stage. First, unlike the re-ranking stage, the ranking stage cannot decide the final recommendation slate. As a result, all of the algorithms with sequential generation function (Wang et al., 2019; Feng et al., 2021a; Wei et al., 2020; Feng et al., 2021b; Bello et al., 2018; Zhuang et al., 2018; Gong et al., 2019), which select the current item based on the previously chosen items, cannot be used at all. Furthermore, many algorithms mentioned above, such as (Ai et al., 2018a; Pei et al., 2019), rely on the item ranking from the previous stage, which is often not available for the ranking stage. Finally, these algorithms are too complex and time-consuming to be practically applied in the ranking stage with much bigger candidate set.

2.2. Privileged Features and Distillation

For recommendation performance, it is essential for us to ensure the consistency between offline training and online inference. However, with this constraint, many discriminative features, which are available at training but not for inference, must be ignored for model training. In (Lopez-Paz et al., 2015; Xu et al., 2020), they define the features only available for training as privileged features. Two models, a student and a teacher model, are trained. Unlike normal model distillation methods (Buciluǎ et al., 2006; Hinton et al., 2015), which input the same feature set to the student and teacher models, the teacher model here utilizes all features while the student model here does not depend on the privileged features. During training, besides the original training labels, knowledge distilled from the teacher model is also used to supervise the training of the student model. After model convergence, only the student model is deployed online to serve the user request. As the student model does not rely on privileged features, the consistency between online and offline is guaranteed.

Taken the slate-wise information as privileged features, we can use PFD (Xu et al., 2020) to indirectly model the mutual influence among the items of a recommendation slate. Compared with SAR, it roughly doubles the training resources. What’s more, SAR can utilize the slate-wise information more effectively and achieve better performance, which is verified in our experiments.

2.3. Multi-Task Learning for Recommender Systems

There are a variety of user interactions in typical recommendation scenario, such as clicking, watch time, liking and commenting. Moreover, single interaction itself are unable to clearly reflect the user satisfaction. Therefore, the recommender system needs to model multiple user behaviors and further combines there predictions to calculate the final utility score. Under such circumstances, we see wide application of multi-task learning techniques as they can solve multiple related tasks simultaneously and improve learning efficiency and prediction accuracy over the single task methods.

We briefly introduce related papers here. Hard parameter sharing is the most intuitive MTL structure. For instance, the ESSM model (Ma et al., 2018a) shares embedding parameters between the tasks of CTR (Click-Through Rate) and CVR (Conversion Rate) for improving the prediction performance of the sparse CVR task. To alleviate the task conflict and negative transfer issue, Zhao et al. (Zhao et al., 2019) extend the Multi-gate Mixture-of-Experts model (MMoE) (Ma et al., 2018b) and apply it to learn multiple ranking objectives in Youtube video recommender systems. PLE (Tang et al., 2020) achieves superior performance for news recommendation by assigning both shared parameters among tasks and task specific parameters.

In section 3, we will describe how to seamlessly integrate SAR with these multi-task learning methods.

3. Methodology

In this section, we introduce the SAR method in detail. First, we formally introduce the problem definition for slate recommendation. Then, we explain the general design of SAR for the ranking stage. Finally, we elaborate on how to enhance the ranking models with the proposed method.

3.1. Slate Recommendation

Given a set of candidate with $N$ items $C=\{c_{k}\}_{1\leq k\leq N}$, the goal of slate recommendation is to recommend a slate of $K$ items $S=\{i_{k}\}_{1\leq k\leq K}\subseteq C$ so as to optimize the overall utility and enhance user experience. We denote the reward of recommending $S$ for user $u$ as $R(S,u)$. Then, the overall slate recommendation problem can be regarded as an combinatorial optimization problem below.

[TABLE]

where $\theta$ is the model parameters for generating $S$ from $C$ given user $u$.

If we suppose the user interaction with one item only depends on the quality and relevance of the corresponding item, then we can factorize the slate reward as a weighted sum of the reward for recommending single items and deduce the equation below.

[TABLE]

where $i_{k}$ the item at position $k$ of the slate, $o_{k}$ is the observation probability for $i_{k}$ and $r(i_{k},u)$ is the reward for recommending a specific item. As $o_{k}$ strictly decreases from $1$ to $K$ for most scenarios, the best slate can be constructed by descending sorting the candidate items’ reward and returning the top $K$ items. This is exactly the underlying rationality of the existing algorithms in the ranking stage.

However, given the well-known mutual influence among the slate items (Bello et al., 2018; Pang et al., 2020; Deng et al., 2018; Zhuang et al., 2018; Pei et al., 2019; Wang et al., 2019; Jiang et al., 2018; Zhao et al., 2017; Gong et al., 2019; Wei et al., 2020; Feng et al., 2021b; Ai et al., 2018a, 2019; Liu et al., 2021; Feng et al., 2021a), it is desirable for us to model the cross impact of slate items to optimize the user experience.

With the multi-stage design of the recommender systems, the matching and pre-ranking stage resort to highly efficient yet coarse-grained models to ensure the coverage rather than precision of the upper stage’s candidate set. Moreover, though there are existing solution for the re-ranking stage to consider the mutual influence of slate items, the re-ranking stage has a much smaller candidate set than the ranking stage. Therefore, this work targets to address the modelling of mutual influence among slate items in the ranking stage to further enhance the end to end recommendation experience by improving the quality of the re-ranking stage’s candidate set.

3.2. General Design of SAR

The ranking stage of recommender systems, which normally selects top tens of items from a pool of hundreds to thousands, originally uses model to estimate the reward score for each pair of user-item and forwards the top items to the re-ranking stage. The reward score may depend on a single objective or multiple objectives. For example, the reward may depend on the user click, item watch time, and other satisfaction-related metrics (e.g., liking and sharing) in the content recommendation application. For multi-objective modelling, we usually leverage multi-task learning approaches (Tang et al., 2020; Ma et al., 2018b) to accurately model multiple user feedback. Furthermore, to compute the overall reward, we need merge the multiple predictions with a function $\Phi$ shown in equation 4 to derive the item’s final reward score for greedy ranking.

[TABLE]

[TABLE]

where $M$ is the number of prediction tasks, $i$ is the target item for score computation and $f_{base}$ is the DNN model to generate prediction for each task. In addition, $\Phi$ is usually a function manually tuned by the engineers to reflect the reward based on the business goals.

From equation 3, we can see the original ranking models does not consider the impact of surrounding items on the current one. Ideally, we should estimate $\{p_{1},p_{2},...,p_{M}\}$ with the equation 5, which takes the recommendation slate $S$ as input. However, we cannot get the full recommendation slate during online serving in the ranking stage.

[TABLE]

As a result, we propose to estimate the slate information based on the user features. Since the recommender systems tend to recommend a slate of items that suits well for the user’s interest, theoretically, we can roughly deduce the slate information with the user features. Specifically, during training, we align the user features and slate features in the transformed latent space, namely $l_{u}$ for user features and $l_{s}$ for slate features. During inference, we firstly estimate $l_{u}$ based on the user features, which is concatenated with the user information and item information to predict $\{p_{1},p_{2},...,p_{M}\}$. In this way, the model inference does not rely on the slate features at all.

3.3. Model Architecture

3.3.1. Baseline Model

For the ranking stage, we usually take two types of features as input, namely user and item features. Traditionally, we also need manual cross features so as to describe feature interactions, which are not so prevalent now because the effective implicit cross within the deep learning models (He and Chua, 2017; Chen et al., 2019; Xiao et al., 2017; He et al., 2017; Cheng et al., 2016) can warrant a better balance between model size and recommendation performance. Both of the two types of features are high-dimensional binary features derived from one-hot encoding of categorical variables (e.g., user and item ids) or discretization of dense variables (e.g., user activity counting). As is displayed in Figure 2(a), for the baseline model, we firstly project each feature to a dense embedding vector with the fully connected embedding layer. Then, the embeddings are concatenated by group to get $e_{i}$ for item features and $e_{u}$ for user features. Then, we concatenate them together and input to the backbone network, which utilizes multi-task algorithms (Tang et al., 2020; Ma et al., 2018b) for multi-objective predictions or simpler algorithms (He et al., 2017; Rendle, 2010; Cheng et al., 2016) for single task prediction.

The whole process can be formally described with the equations below.

[TABLE]

[TABLE]

[TABLE]

where $Embedding_{u}$ and $Embedding_{i}$ are the embedding and concatenation operation for the user features and item features respectively. And $Backbone$ denotes the backbone network.

3.3.2. Introduced Components of SAR

Please refer to Figure 2(b) and 2(c) for the model architecture of SAR. First, with SAR, besides the aforementioned two types of features, we also add slate-wise features, such as the sequence of item ids and categories in the recommendation slate. First, these slate-wise features are projected to low dimensional representation with an embedding layer and then concatenated together to get $e_{s}$ with equation 9.

[TABLE]

Second, we add two encoder networks, namely $Encoder_{u}$ and $Encoder_{s}$, for the user features and slate-wise features respectively to get the transformed latent space to be aligned with. The slate encoder network denoted by Equation 10 takes $e_{s}$ and $e_{u}$ as input and generates the intermediate representation vector ($l_{s}$) representing the slate information with personalized weight for different slate items. Moreover, the user encoder network denoted by Equation 11 accepts $e_{u}$ as input and produces a representation vector ($l_{u}$) to be aligned with $l_{s}$ during training.

[TABLE]

[TABLE]

Additionally, we introduce the decoder network as is described in 12. Besides the auxiliary input of $e_{u}$, it also accepts $l_{u}$ as input for inference or $l_{s}$ as input for training. The decoder network extracts valuable information from the output latent space of the encoder networks for the specific user and tries to enhance the overall recommendation performance.

[TABLE]

where $l$ can be $l_{u}$ or $l_{s}$ based on the status of inference or training. And $d$ is the decoding result.

Finally, the decoded information ($d$) together with $e_{u}$ and $e_{i}$ is then fed to the backbone networks to generate more accurate predictions for the tasks.

[TABLE]

For the slate encoder network ($Encoder_{s}$), because of the sequential nature of the item permutation in the slate, we may firstly process the sequence of item features with the techniques of sum pooling, target attention (Zhou et al., 2018), LSTM(Greff et al., 2016) or transformer (Vaswani et al., 2017), to extract essential information to be concatenated with $e_{u}$ for further computation with the MLP network within $Encoder_{s}$. For all the other newly introduced modules, we just consider simple MLP networks.

3.3.3. SAR Training

For SAR training displayed in Figure 2(b), we need all the aforementioned components, including the slate-wise features and the slate encoder networks. The encoding result of the user encoder network and the slate encoder network is aligned with the supervision of the similarity loss. Furthermore, together with the auxiliary input of $e_{u}$, the decoder network tries to extract valuable information from the output of the slate encoder network for the specific user, which is further concatenated with $[e_{i},e_{u}]$ and fed to the backbone network for prediction.

3.3.4. SAR Inference

For SAR inference displayed in Figure 2(c), we no longer need the slate-wise features and the slate encoder network. The result of the user encoder network rather than the slate encoder network is fed to the decoder network for further processing. The prediction performance is guaranteed with the proper supervision from the similarity loss during training, which enforces little gap and maximum similarity between the outputs of the user encoder network and the slate encoder network.

3.4. Joint Loss Optimization

In multi-task learning, a common formulation of joint loss is the weighted sum of the losses for individual tasks. With our method, we just add an auxiliary similarity loss to ensure the similarity between the outputs of the slate encoder network and user encoder network. Then, for multi-task learning with $M$ tasks, given input features $X$ and task specific labels $Y_{m},m=1,2,...,M$, we learn the model parameters by minimizing the aggregated loss of equation 14. $\omega_{m}$ and $L_{m}$ are the weight and loss for the task $m$ respectively. For $Loss_{sim}$, we use the L2 loss to minimize the Euclidean distance of the results of the user encoder network ($l_{u}$) and the slate encoder network ($l_{s}$). And $\lambda$ is the loss weight for the similarity loss.

[TABLE]

It is important for $\lambda$, the weight of the similarity loss, to be tuned and properly set. On the one hand, the model training may not converge with a too large value. Instead, if it is set too small, there will be unfavorable inconsistencies between training and inference. If the model performance is over sensitive to this parameter, there will be difficulties for SAR to be practically applied. We will perform sensitivity test and report the results in our experiments.

4. Experiments

In this section, we conduct offline and online experiments with the aim of answering the following questions:

• RQ1

Compared with the baseline and PFD (Xu et al., 2020), what is the offline performance of SAR on different backbone networks?

• RQ2

Can SAR achieve additional online gains if deployed on top of the state of the art re-ranking method for slate recommendation?

• RQ3

What is the impact of the similarity loss’s weight on model performance? How sensitive does the value of $\lambda$ impact the overall recommendation quality?

• RQ4

Can the slate encoder and user encoder network generate similar embedding vectors so as to ensure the serving performance when the similarity loss’s weight is properly set?

4.1. Experimental Settings

4.1.1. Datasets

We experimented222We open source the code at: https://github.com/BestActionNow/Slate_Aware_Ranking on two real-world datasets. The first one is the public MovieLens-1M333http://www.grouplens.org/datasets/movielens dataset consisting of 1 million ratings from 6000 users on 4000 movies. Considering the slate information is absent in MovieLens-1M, we followed the practice of (Liu et al., 2021; Jiang et al., 2018) to split user rating sessions into slates of size 20 and take the ratings of 4-5 as positive and 1-3 as negative. The data were randomly separated into training set, validation set and test set by the ratio of 8:1:1.

Moreover, we sampled an industrial dataset from 9 days’ user logs of our industrial news recommender systems. There are about 30 million users, 2 million items and 1 billion samples in the datasets. In our application, for each user interaction, the system recommends a slate of 10 items, whose ids are used as the slate feature of each exposed item. And our main prediction tasks are the binary classification task of click through rate with cross entropy loss and the regression task of item watch time after click with huber loss. We used the samples in the first 7 days for training. The samples of the 8th day were used for validation and hyper-parameter tuning. And samples of the last remaining day were for testing.

4.1.2. Experimental Models

For the slate encoder network at Figure 2, we experiment several variants of slate feature transformation to study the impact of different slate encoding structures. With these variants, we would like to understand whether the slate’s sequential information and the proper attention weights for different slate items are beneficial for the overall recommendation performance. For all of these variants, we concatenate the above result with $e_{u}$ and input to the MLP component of the slate encoder network.

• •

$SAR_{SumPool}$: We perform sum pooling with the embeddings of the slate items to obtain the slate embedding. For this configuration, neither sequential information nor relative importance of slate items can be captured.

• •

$SAR_{LSTM}$: We input the embeddings of the slate items to a one layer LSTM network to get the slate embedding. For this configuration, the sequential information of slate items is kept. But we do not compute the relative weights of the slate items based on the target item, whose ranking scores are to be predicted.

• •

$SAR_{Attn}$: Following (Zhou et al., 2018), we single out the target item as query to get the attention weight over the slate items. Please note that the target item belongs to the set of slate items. Then, weighted sum is executed to get the slate embedding. In this way, we get the relative weights of the slate items favorable for the target item but lose the slate’s sequential information.

For the user encoder, slate encoder and decoder network of SAR, we use MLP structure of [$dim$, $dim$] where $dim$ is set to 16 and 128 for MovieLens-1M and the industrial data respectively. For the backbone network at Figure 2, PLE (Tang et al., 2020) is utilized for the industrial dataset. And multiple backbone networks, including FM(Rendle, 2010), Wide&Deep(Cheng et al., 2016) and NCF(He et al., 2017), are tested for MovieLens-1M to verify the wide compatibility of our method.

And we also take the slate-wise features as privileged features, which are discriminative but only available during training, and test the performance of privileged feature distillation ($PFD$) (Xu et al., 2020). For fairness, we would like the teacher model of PFD to have the same capacity as the model for SAR training. Thus, after disabling the similarity loss, we use the model structure of Figure 2(b) with the $SAR_{Attn}$ slate encoder network as teacher model and the model structure of Figure 2(a) as student model.

For all the models, we use Adam (Kingma and Ba, 2014) optimizer. We tune the learning rate and other hyper-parameters for each model separately.

4.1.3. Evaluation Metrics

From equation 13 and 8, we can see that the main task of the ranking model is to predict one or several user feedback, such as clicking, watch time, liking and sharing. And the merge function $\Phi$ in Equation 4, which is a function manually tuned to reflect the reward based on the business goals, assumes that the ranking model can estimate accurate interaction probabilities for binary classification tasks (e.g. liking and sharing) and absolute values for regression tasks (e.g. watch time).

As a result, instead of the ranking metrics, such as Normalized Discounted Cumulative Gain (NDCG) (Wang et al., 2013) and Mean Reciprocal Rank (MRR) (Wikipedia contributors, 2022a), we use the metrics of Area Under the ROC Curve (AUC) (Flach et al., 2011) for classification tasks and Mean Absolute Error (MAE) (Wikipedia contributors, 2022c) for regression tasks. Please note that many other recommendation literature, such as (Tang et al., 2020), also use similar metrics. For AUC, a bigger value indicates better performance. While, for MAE, it is the smaller the better.

4.2. Offline Performance Comparison(RQ1)

As mentioned above, we use AUC and MAE as the evaluation metrics for the tasks of classification and regression respectively. Please note that a 0.1$\%$ relative gain over baseline means significant performance improvement for both metrics in our scenario.

4.2.1. Industrial Dataset

The baseline’s architecture is shown in Figure 2(a) with PLE (Tang et al., 2020) as its backbone network. For PFD, the student model is of the same structure as the baseline. But its student model is trained with the supervision from both the soft labels of the teacher model and the original hard labels. For the teacher model of PFD, after removing the similarity loss, we use the model structure of Figure 2(b) with $SAR_{Attn}$ as the slate encoder network. For SAR, we use different model structures for training and inference, which is described in section 3. For the backbone network of SAR, it is of the same structure as the backbone network of the baseline except for the input layer, which is of additional dimension with the introduction of $d$ in Equation 13.

From Table 2, we can see that our method achieves competitive performance. Compared with baseline, $SAR_{Attn}$ has 0.43$\%$ gain on AUC and 0.2$\%$ decrease on MAE. $SAR_{Attn}$ performs better than $PFD_{student}$ with 0.33$\%$ gain on AUC and 0.11$\%$ decrease on MAE. In other words, $SAR$ significantly outperforms $PFD$ with about half training resources.

Among the variants of $SAR$, $SAR_{Attn}$ shows some marginal gains over $SAR_{LSTM}$ and $SAR_{SumPool}$. Based on our test, the gain is stable for multiple runs. Moreover, $SAR_{LSTM}$ and $SAR_{SumPool}$ have roughly the same performance. Therefore, it seems the computed relative weights of the slate items are useful for model prediction.

4.2.2. MovieLens-1M

For MovieLens-1M, we compare $SAR_{Attn}$, which exhibits the best performance for the industrial data, with baseline and $PFD$ on three backbone networks. For PFD, the student model is still of the same structure as the baseline model shown in Figure 2(a). And the teacher model of PFD uses the model structure of Figure 2(b) with $SAR_{Attn}$ as the slate encoder network after disabling the similarity loss. For SAR, we still use different model structures for training and inference shown at Figure 2(b) and Figure 2(c) respectively. For this experiment, we test three different backbone networks listed in table 3 to study SAR’s performance gain over different backbone networks.

The results are summarized in Table 3. Compared with the baseline, PFD shows gain for the backbone network of Wide&Deep and NCF but neural results for FM. Moreover, SAR shows consistent gains over PFD and the baseline model. Overall, the results of Movie-Lens 1M demonstrate the effectiveness and wide compatibility of $SAR$.

4.3. Online A/B Testing(RQ2)

To verify the effectiveness of our proposed algorithm, we served $SAR_{Attn}$ model online in the ranking stage, which accepts hundreds of candidates and outputs the top 50 items to the re-ranking stage, of our content recommender systems. In the re-ranking stage, we already deployed the state of the art generator and evaluator algorithms similar to (Feng et al., 2021a) for slate recommendation. We randomly distributed online users to two buckets with the baseline ranking model or SAR and evaluated the performance for seven days to report our main online metrics of total page view per person (PV) and app stay time per person (ST). As is shown in table 4, SAR achieved significant gains over the baseline by 0.9$\%$ for page view and 0.745$\%$ for stay time. Based on the results from our A/B test platform, the p-value is below 0.05 for one-tailed t-test for both metrics mentioned above.

Unlike search engine, the user’s intention is not so focused in recommendation scenario. Thus, after implicitly modelling the mutual influence of the slate items with SAR, the recommendation results should cover more interests of the user in a recommendation slate guided by overall utility optimization, thereby exhibiting enhanced diversity. We are concerned about diversity because of the following reasons. On the one hand, diverse recommendation is helpful for immediate user satisfaction. On the other hand, in all likelihood, the system can understand more multi-faceted user interests so as to ensure even better recommendation performance in the future. Furthermore, diverse recommendation will encourage the authors to provide a wide variety of contents, which is beneficial for the long term success of the ecosystem. To verify SAR’s effectiveness on diversity, we extracted one day’s online data and analyzed the final recommendation results. We computed the exposed items’ gini index (Wikipedia contributors, 2022b) metrics on item id and item category. From table 5, compared with the baseline traffic, we can see lower gini index indicating better diversity for the final recommendation results, especially for the metrics of item category.

4.4. Impact of Similarity Loss Weight(RQ3)

We further studied the impact of the weight of similarity loss on model performance in the industrial dataset. First, following the results of previous experiments, we fixed the loss weights of the CTR task and watch time task to the optimal values. Then, to report the model performance of $SAR_{Attn}$, we tested different weight ratio of the similarity loss relative to the CTR task.

According to Figure 3, the best model performance is achieved when the similarity loss’s weight is equal to the CTR task. Moreover, $SAR_{Attn}$’s performance is not too sensitive to the change of the similarity loss weight. So hyper-parameter tuning is of manageable difficulty and SAR can be practically applied to boost the recommendation performance.

4.5. Encoder Network Embedding Distribution Analysis on Industrial Dataset(RQ4)

We visualized the embedding distribution of the User and Slate Encoder Networks with t-SNE (Van der Maaten and Hinton, 2008) in figure 4(a), 4(c) and 4(d) for the best performing $SAR_{Attn}$ model with equal loss weights for the similarity task and the CTR task. From figure 4(a), we can see the points fall into two clusters and the embeddings from the two encoders show similar distribution. With figure 4(c) and 4(d), it is obvious that the distribution density match well in each dimension.

In figure 4(b), we randomly sampled six output pairs from these two encoder networks. For each pair, we got the encoding results from the two encoder networks for the same instance. And we plotted with different colors for points from different networks. The points from the same pair were of the same shape. For each pair, the two points resided in the same cluster and were close to each other.

In summary, the User Encoder network and Slate Encoder Network can generate outputs with consistent distributions, which ensures the consistency between training and inference and explains the effectiveness of our proposed method.

5. Conclusion

In this paper, we propose an algorithm for the ranking stage of recommender systems to predict user behaviors more accurately by implicitly modelling the contextual items’ impact. We perform thorough offline experiments and online A/B testing to prove that it significantly enhances the recommendation relevance and diversity by ensuring the quality of the re-ranking stage’s candidates. Moreover, additional experiments are carried out to study the similarity loss weight’s impact and understand why $SAR$ works. In future, we will try to further improve the recommendation performance by designing more effective models in the ranking and re-ranking stage to tackle the issue of cross item influence. Furthermore, though the matching and pre-ranking stage care more about coverage than precision, it is interesting to investigate whether addressing the aforementioned issue in these two layers can boost the overall recommendation quality. Finally, as it is essential to properly handle position bias for precise user preference modelling (Ren et al., 2022; Joachims et al., 2017; Ai et al., 2018b), we plan to study how to more accurately estimate the context items’ impact on target items at different positions.