Conformative Filtering for Implicit Feedback Data

TL;DR

This paper introduces Conformative Filtering, a novel recommendation method that clusters users by taste to better infer item relevance from implicit feedback, addressing the challenge of non-consumption not always indicating disinterest.

Contribution

The paper proposes a new clustering-based approach, Conformative Filtering, utilizing hierarchical latent tree analysis to improve recommendations from implicit feedback data.

Findings

CoF outperforms baseline methods on real-world datasets.

Clustering users by taste improves recommendation accuracy.

The method effectively handles the ambiguity of non-consumption in implicit feedback.

Abstract

Implicit feedback is the simplest form of user feedback that can be used for item recommendation. It is easy to collect and is domain independent. However, there is a lack of negative examples. Previous work tackles this problem by assuming that users are not interested or not as much interested in the unconsumed items. Those assumptions are often severely violated since non-consumption can be due to factors like unawareness or lack of resources. Therefore, non-consumption by a user does not always mean disinterest or irrelevance. In this paper, we propose a novel method called Conformative Filtering (CoF) to address the issue. The motivating observation is that if there is a large group of users who share the same taste and none of them have consumed an item before, then it is likely that the item is not of interest to the group. We perform multidimensional clustering on implicit feedback data using hierarchical latent tree analysis (HLTA) to identify user `tastes' groups and make recommendations for a user based on her memberships in the groups and on the past behavior of the groups. Experiments on two real-world datasets from different domains show that CoF has superior performance compared to several common baselines.

Tables

Table 1. Table 1: User clusters identified by the latent variables Z 13 subscript 𝑍 13 Z_{13} and Z 1147 subscript 𝑍 1147 Z_{1147} . High percentages of the users in the cluster Z 13 = s 1 subscript 𝑍 13 subscript 𝑠 1 Z_{13}=s_{1} have watched the three movies Armageddon , Golden Eye and Con Air . Hence, the cluster is regarded as a user group with a taste for the three movies. Similarly, the cluster Z 1147 = s 1 subscript 𝑍 1147 subscript 𝑠 1 Z_{1147}=s_{1} is regarded as a user group with a taste for the three movies The Great Muppet Caper , Pete’s Dragon and The Muppets take Manhattan .

	$Z_{13}=s_1$	$Z_{13}=s_0$
Action-Adventure-Thriller	(0.21)	(0.79)
$\mathrm{𝙰𝚛𝚖𝚊𝚐𝚎𝚍𝚍𝚘𝚗}$	0.610	0.055
$\mathrm{𝙶𝚘𝚕𝚍𝚎𝚗}\_\mathrm{𝙴𝚢𝚎}$	0.588	0.013
$\mathrm{𝙲𝚘𝚗}\_\mathrm{𝙰𝚒𝚛}$	0.635	0.014

Table 2. Table 2: Parameters selected by validation.

	Movielens20M		Ta-feng
CoF	$l$=1,$H$=5		$l$=1,$H$=40
WRMF	$F$=40,$\lambda$=${10}^{-2}$		$F$=10,$\lambda$=10
BPRMF	$F$=80,$\lambda$=${10}^{-2}$		$F$=80,$\lambda$=${10}^{-4}$
Ocular	$F$=120,$\lambda$=80		$F$=60,$\lambda$=120
SLIM	N.A.		$k$=500,$\lambda$=${10}^{-2}$,$\beta /2$=10
GBPR	$F$=160,$\lambda ={10}^{-4}$, $\left|G\right|=10$,$\rho =0.2$		$F$=20,$\lambda$=0.1, $\left|G\right|=4$,$\rho =0.4$

Table 3. Table 3: The AUC for each recommender is shown. CoF outperforms other methods.

	BPRMF	WRMF	CoF	Ocular	SLIM	GBPR
Ta-feng	0.74977	0.71316	0.7793	0.63653	0.68321	0.71117
ML-20M	0.87289	0.85258	0.88816	0.84879	N.A.	0.80367

Equations

$$\sum _{(u,i)\in \mathcal{C}}(r_{ui}-{\hat{r}}_{ui}{)}^2+\text{mbox}regularizationterms.$$

$$\sum _{(u,i)\in \mathcal{C}}(1-{\hat{r}}_{ui}{)}^2+\sum _{(u,j)\in \mathcal{U}}(0-{\hat{r}}_{uj}{)}^2+\text{mbox}regularizationterms.$$

$$\sum _{(u,i)\in \mathcal{C}}(1-{\hat{r}}_{ui}{)}^2+\sum _{(u,j)\in \mathcal{U}}{c}_{uj}(0-{\hat{r}}_{uj}{)}^2+\text{mbox}regularizationterms,$$

$$\sum _{(u,i)\in \mathcal{C}}(1-{\hat{r}}_{ui}{)}^2+\sum _{(u,j)\in {\mathcal{U}}^{\prime }}(0-{\hat{r}}_{uj}{)}^2+\text{mbox}regularizationterms,$$

$$-\sum _{(u,i)\in \mathcal{C}}\log (|1-e^{-{\hat{r}}_{ui}}|)-\sum _{(u,j)\in \mathcal{U}}\log (|0-e^{-{\hat{r}}_{uj}}|)+\text{mbox}regularizationterms.$$

$$\sum _u\sum _{i\in {\mathcal{C}}_u}\sum _{j\in {\mathcal{U}}_u}-\log \sigma ({\hat{r}}_{ui}-{\hat{r}}_{uj})+\text{mbox}regularizationterms,$$

$$\varphi (i|G_k,\mathcal{D})=\frac{{\sum }_u\mathbb{I}(i|u,\mathcal{D})P(G_k|u,m)}{{\sum }_{u}P(G_k|u,m)},$$

$${\mathbf{f}}_i=(\varphi (i|G_1,{\mathcal{D}}_H),\dots ,\varphi (i|G_K,{\mathcal{D}}_H)).$$

$${\mathbf{f}}_u=(P(G_1|u,m),\dots ,P(G_K|u,m)).$$

$${\hat{r}}_{ui}=\sum _{k=1}^K\varphi (i|G_k,{\mathcal{D}}_H)P(G_k|u,m).$$

Full text

1 Introduction

With the advent of the online marketplace, an average user is presented with an un-ending choice of items to consume. Those could be books to buy, web-pages to click, songs to listen, movies to watch, and so on. Online stores and content providers no longer have to worry about shelf space to display their items. However, too much choice is not always a luxury. It can also be an unwanted distraction and makes it difficult for a user to find the items she desires. It is necessary to automatically filter a vast amount of items and identify those that are of interest to a user.

Collaborative filtering (CF) [7] is one commonly used technique to deal with the problem. Most research work on CF focuses on explicit feedback data, where ratings on items have been previously provided by users [13]. Items with high ratings are preferred over those with low ratings. In other words, items with high ratings are positive examples, while those with low ratings are negative examples. Unrated items are missing data.

In practice, one often encounters implicit feedback data, where users did not explicitly rate items [18]. Recommendations need to be made based on user activities such as clicks, page views, and purchase actions. Those are positive-only data and contain information regarding which items were consumed. There is no information about the unconsumed items. In other words, there are no negative examples. The problem is hence called one class collaborative filtering (OCCF) [20].

In previous works, the lack of negative examples in OCCF is addressed by adopting one of the following four strategies with respect to each user: (1) Treat unconsumed items as negative examples [19]; (2) Treat unconsumed items as negative examples with low confidence [11]; (3) Identify some unconsumed items as negative examples using heuristics [20]; (4) Assume the user prefers consumed items over unconsumed items [25]. We refer to the strategies as the unconsumed as negative (UAN), UAN-with-low-confidence, UAN-with-chance and consumed preferred over unconsumed (CPU) assumptions respectively.

All the assumptions are problematic. The UAN assumption is in contradiction with the very objective of collaborative filtering — to identify items that might be of interest to a user among those she did not consume before. Moreover, if we assume a user does not like two items to exactly the same degree, then theoretically there is 50% chance that she would prefer the next item she chooses to consume to the last item she consumed.

In this paper, we adopt a new assumption: If there is a large group of users who share the same taste and none of them have consumed an item before, then the item is not of interest to the group. By a taste we mean the tendency to consume a certain collection of items such as comedy movies, pop songs, or spicy food. We call our assumption the group UAN assumption because it is with respect to a user group. In contrast, we refer to the first assumption mentioned above as the individual UAN because it is with respect to an individual user. Group UAN is more reasonable than individual UAN because there is less chance of treating unawareness as disinterest.

We identify user taste groups by performing multidimensional clustering using hierarchical latent class analysis (HLTA) [2]. HLTA can detect sets of items that tend to be co-consumed in the sense users who consumed some of the items in a set often also consumed others items in the set, albeit not necessarily at the same time. HLTA can also determine the users who showed the tendency to consume the items in a co-consumption set. Those users make up a taste group. To make recommendation for a user, we consider her memberships in the taste groups and past behaviors of those groups. We call this method Conformative Filtering (CoF) because a user is expected to conform to the behaviors of the groups she belongs to.

The main contributions of this paper include:

Proposing an intuitively appealing strategy, namely group UAN, to deal with the lack of negative examples; 2. 2.

Proposing a novel framework for OCCF, i.e., CoF, that is based on this assumption; 3. 3.

Using HLTA, an algorithm proposed for text analysis, to solve a fundamental problem in collaborative filtering;

The empirical results show that CoF significantly outperforms the state-of-the-art OCCF recommenders in predicting the items that users want to consume in the future. In addition, the latent factors in CoF are more interpretable than those in matrix factorization methods.

2 Related Work

In the model-based approach to collaborative filtering, the goal is to find a feature vector $\bf{f}_{u}$ for each user $u$ and a feature vector $\bf{f}_{i}$ for each item $i$, and predict the rating of user $u$ for item $i$ using the inner products of the two vectors, i.e., $\hat{r}_{ui}=<\bf{f}_{u},\bf{f}_{i}>$. The dimension of $\bf{f}_{u}$ and $\bf{f}_{i}$ is usually much smaller than the number of users and the number of items.

Let $\mathcal{C}$ be the set all consumption pairs, i.e., user-item pairs $(u,i)$ such that $u$ consumed $i$ before. The complement $\mathcal{U}$ of $\mathcal{C}$ consists of non-consumption pairs. In the case of explicit feedback data, we have a rating $r_{ui}$ for each pair $(u,i)\in\mathcal{C}$. It is the rating for item $i$ given by user $u$ and its possible values are usually the integers between 1 and 5. The feature vectors can obtained by minimizing the following loss function:

[TABLE]

In the literature, this is known as the matrix factorization (MF) method [14] because $[\bf{f}_{u}]^{\top}[\bf{f}_{i}]$ is an approximate low-rank factorization of the user-item matrix $[r_{ui}]$.

For implicit feedback data, researchers usually set $r_{ui}=1$ for consumption pairs $(u,i)\in\mathcal{C}$. There is no information about $r_{uj}$ for non-consumption pairs $(u,j)\in\mathcal{U}$. In this case, minimizing equation (2) would lead to non-sensible solutions. Several methods have been proposed to solve the problem. We briefly review them below. Regularization terms and constraints are ignored for simplicity.

The sparse linear method (SLIM) [19] makes the individual UAN assumption and sets $r_{uj}=0$ for all $(u,j)\in\mathcal{U}$. It minimizes:

[TABLE]

In addition, it lets $\bf{f}_{u}$ be the binary vector over items that represents past consumptions of user $u$, and it only finds $\bf{f}_{i}$.

The weighted regularized MF (WRMF)[11, 20] algorithm makes the UAN-with-low-confidence assumption and minimizes:

[TABLE]

where $0\leq c_{uj}\leq 1$ for all $(u,j)\in\mathcal{U}$. The values of the weights $c_{uj}$ indicate the confidence in treating the non-consumption pairs as negative examples.

The negative sampling method [20] makes the UAN-with-chance assumption and minimizes:

[TABLE]

where $\mathcal{U}^{\prime}$ is a randomly sampled subset of $\mathcal{U}$.

The overlapping co-cluster recommendation (Ocular) algorithm [9] minimizes:

[TABLE]

This loss functions gives large penalty if ${\hat{r}}_{ui}$ is close to 0 for consumption pairs $(u,i)$ and small penalty if ${\hat{r}}_{uj}$ is close to 1 for non-consumption pairs $(u,j)$. There is stronger “force” pushing ${\hat{r}}_{ui}$ toward 1 and weaker “force” pushing ${\hat{r}}_{uj}$ toward 0. So, ocular is implicitly making the UAN-with-low-confidence assumption.

The Bayesian personalized ranking MF (BPRMF) [25] algorithm makes the CPU assumption and minimizes:

[TABLE]

where $\mathcal{C}_{u}=\{i|(u,i)\in\mathcal{C}\}$, $\mathcal{U}_{u}=\{j|(u,j)\in\mathcal{U}\}$, and $\sigma$ is the sigmoid function. The penalty for a user $u$ is small if the predicted scores ${\hat{r}}_{ui}$ for the consumed items are large relative to the predicted scores $r_{uj}$ for the unconsumed items.

Various extensions of the aforementioned methods have been proposed. SLIM has been extended by [4, 5, 15], WRMF has been extended by [8, 24, 27], and BPRMF has been extended by [10, 21, 22, 26].

Clustering algorithms have also been applied to CF. The k-means algorithm and hierarchical clustering have been used to group either users or items to reduce the time complexity of CF methods such as user-kNN and item-kNN [1, 13]. Co-clustering has been used to identify user-item co-clusters so as to model user group heterogeneity [9, 29, 30]. In [29, 30], the authors find multiple sub-matrices (co-clusters) of the user-item matrix, apply another CF method on each sub-matrix, and then aggregate the results. In ocular [9], the authors first obtain feature vectors $f_{u}$ and $f_{i}$, and then use those vectors to produce user-item co-clusters for the sake of interpretability of the results. The notion of user groups is used in an extension of BPRMF called GBPR [22]. In GBPR a group of users is formed for each consumption pair $(u,i)$, and it consists of a few randomly selected other users who also consumed the item $i$ before. In CoF a user taste group is determined based on a set of items that tend to be co-consumed, and it is typically quite large. However, none of the aforementioned methods use the (user, item, or user-item) clusters obtained to deal with the lack of negative preference in implicit feedback data.

3 User taste Group Detection using HLTA

When applied to implicit feedback data, HLTA111https://github.com/kmpoon/hlta. learns models such as the one shown in Figure 1, which was obtained from the Movielens dataset. Movielens is an explicit feedback dataset. It was turned into an implicit feedback dataset by ignoring the item ratings222Movielens is used for illustration since movies genres are easier to interpret..

The model is a tree-structured Bayesian network, where there is a layer of observed variables at the bottom, and multiple layers of latent variables on top. It is called a hierarchical latent tree (HLTM) model [2, 17]. The model parameters include a marginal distribution for the root333When there are multiple latent variables at the top level, arbitrarily pick one of them as the root. and a conditional distribution for each of the other nodes given its parent. The product of the distributions defines a joint distribution over all the variables.

In this paper, all the variables are assumed to be binary. The observed variables indicate whether the items were consumed by a user. For example, the value of the variable Mulan for a user is 1 if she watched the movie before, and 0 otherwise. Note that here the value 0 means non-consumption, not disinterest.

The latent variables are introduced during data analysis to explain co-consumption patterns detected in data. For example, the fact that the variables Armageddon, Golden Eye and Con Air are grouped under $Z_{13}$ indicates that the three movies tend to be co-consumed, in the sense users who watched one of them often also watched the other two. The pattern is explained by assuming that there is a taste, denoted by $Z_{13}$, such that users with the taste tend to watch the movies and users without it do not tend to watch the movies. Similarly, $Z_{14}$ explains the co-consumption of The Seige, Mask of Zorro, Daylight and The River Wild. $Z_{22}$ indicates that the patterns represented by $Z_{13}$ and $Z_{14}$ tend to co-occur.

HLTMs are a generalization of latent class models (LCMs) [12], which is a type of finite mixture models for discrete data. In a finite mixture model, there is one latent variable and it is used to partition objects into soft clusters. Similarly, in an HLTM, each latent variable partitions all the users into two clusters. Since there are multiple latent variables, multiple partitions are obtained. In this sense, HLTMs are a tool for multidimensional clustering [3, 16, 31].

Information about the partition given by $Z_{13}$ is shown in Table 1. The first cluster $Z_{13}=s_{1}$ consists of 21% of the users. High percentages of the users in the cluster have watched the three movies Armageddon, Golden Eye and Con Air. So, they have a taste for them. In contrast, few users in the second cluster $Z_{13}=s_{0}$ have watched these movies and hence they do not possess the taste.

Similarly, $Z_{14}$ identifies another group of users with a taste for the movies The Seige, Mask of Zorro, Daylight and The River Wild. $Z_{14}$ and $Z_{13}$ are grouped under $Z_{22}$ in the model structure, which indicates that the two tastes tend to be co-possessed, and $Z_{22}$ identifies the users who tend to have both tastes.

4 Conformative Filtering

Suppose we have learned an HLTM $m$ from an implicit feedback dataset and suppose there are $K$ latent variables on the $l$-th level of the model, each with two states $s_{0}$ and $s_{1}$. Denote the latent variables as $Z_{l1},\ldots,Z_{lK}$. They give us $K$ user taste groups $Z_{l1}=s_{1},\ldots,Z_{lK}=s_{1}$, which will sometimes be denoted as $G_{1},\ldots,G_{K}$ for simplicity. In this section, we explain how these user taste groups can be used for item recommendation.

4.1 User Group Characterization

A natural way to characterize the preferences of a user group for items is to aggregate past behaviors of the group members. The issue is somewhat complex for us because our user groups are soft clusters. Let $\mathbb{I}(i|u,{\mathcal{D}})$ be the indicator function which takes value 1 if user $u$ has consumed item $i$ before, and 0 otherwise. We determine the preference of a taste group $G_{k}$ (i.e., $Z_{lk}=s_{1}$) for an item $i$ using the relative frequency that the item was consumed by users in the group, i.e.:

[TABLE]

where $P(G_{k}|u,m)$ is the probability of user $u$ belonging to group $G_{k}$, and the summations are over all the users who consumed item $i$ before.

Note that $\phi(i|G_{k},{\mathcal{D}})=0$ if no users in $G_{k}$ have consumed the item $i$ before. In other words, we assume that a group is not interested in an item if none of the group members have consumed the item before.

There is an important remark to make. The reason we determine the preferences of a user group $G_{k}$ is that we want to predict future behavior of the group. As such, we might want to base the prediction on recent behaviors of the group members instead of their entire consumption histories. For example, we might want to choose to use a subset ${\mathcal{D}}_{H}$ of the data that consists of only the latest $H$ consumptions for each user. We will empirically investigate this strategy and will show that the choice of $H$ has an impact on the quality of item recommendations.

4.2 Item Recommendation

Having characterized the user taste groups, we now give feature vectors for items and users. We characterize item $i$ using a vector where the $k$-th component is the relative frequency that it was consumed by members of group $G_{k}$, i.e.,

[TABLE]

Note that ${\mathcal{D}}_{H}$ is used instead of ${\mathcal{D}}$, which means that the latent representation is obtained from the $H$ most recent consumptions of users.

We characterize user $u$ using a vector where the $k$-th component is the probability that user $u$ belongs to the group $G_{k}$, i.e.,

[TABLE]

The latent representations require the computation the posterior probabilities $P(G_{k}|u,m)=P(Z_{lk}=s_{1}|u,m)$ for $k=1,\ldots,K$. Because $m$ is a tree-structured model, all the posterior probabilities can be computed by propagating messages over the tree twice [23]. It takes time linear in the number of variables in the model, and hence linear in the number of items.

We use the inner product of the two vectors $\bf{f}_{i}$ and $\bf{f}_{u}$ as the predicted score $\hat{r}_{ui}$ for the user-item pair $(u,i)$, i.e.,

[TABLE]

To make recommendations for a user $u$, we sort all the items $i$ in descending order of the predicted scores $\hat{r}_{ui}$, and recommend the items with the highest scores.

4.3 Discussions

Matrix factorization (MF) is often used in collaborative filtering to map items and users to feature vectors in the same Euclidean space. The components of the vectors are called latent factors, which are not be confused with latent variables in latent tree models.

CoF differs fundamentally from MF. The latent factors in MF are obtained by factorizing the user-item matrix and they are not interpretable. In contrast, the latent factors in CoF are characteristics of user taste groups and they have clear semantics.

CoF naturally incorporates the group UAN assumption, which is the most reasonable assumption to deal with the lack of negative examples to date. In contrast, MF has only been extended to incorporate the individual UAN assumption and its variants. It cannot incorporate the group UAN assumption because there is no notion of user groups.

In addition, CoF has a desirable characteristic that is not shared by MF. It considers the entire consumption histories of the users when grouping them and uses only recent user behaviors when predicting what the groups would like to consume in the future. Consumption behaviors long ago are useful when identifying similar users. However, they might not be very useful when predicting what the users would like to consume in the future. Actually, they might be misleading.

5 Experiments

We performed experiments on two real-world datasets to compare CoF with five baselines using two evaluation metrics.

5.1 Datasets

The datasets used in our experiments are Movielens20M444https://grouplens.org/datasets/movielens/20m/ and Ta-feng555www.bigdatalab.ac.cn/benchmark/bm/dd?data=Ta-Feng. Movielens20M contains ratings given by users to the movies they watched. It is an explicit feedback dataset and was converted to implicit feedback data by keeping all the rating events and ignoring the rating values. Ta-feng contains purchase events at a supermarket, where each event is a customer-item pair and a checking-out action by a customer involves multiple events.

Statistics about the datasets are as follows:

[TABLE]

Each dataset is comprised of (user, item, time-stamp) tuples. Following [28], we split each dataset into training, validation and test sets by time. This is so that all the training instances came before all the testing instances, which matches real-world scenarios better than splits that do not consider time. We tested on several splits and the results were similar. In the following, we will only report the results on the split with 70% of the data for training, 15% for validation and 15% for test.

5.2 Baselines

In the Related Work section, we discussed five representative OCCF methods, namely WRMF, BPRMF, ocular (co-clustering), GBPR (group based) and SLIM (model based neighborhood method). They were all included in our experiments.666SLIM failed to finish a single run in one week on the Movielens20M dataset during validation, therefore its performance is not reported on this dataset. The implementation of the original authors was used for ocular, and the LibRec implementations777https://www.librec.net/index.html and the MyMediaLite implementations [6] were used for all other baselines.

All the algorithms require some input parameters. For the baselines, we tuned their parameters on the validation set through grid search, as is commonly done in the literature. The best parameters were chosen based on recall@$R$. The details of the parameters chosen can be found in Table 2. The key parameters are: $F$ — the number of latent factors; $\lambda$, $\beta/2$ — weights for regularization terms; $k$ — the size of neighborhood; $|G|$ — group size; $\rho$ — tuning parameter. We refer the reader to the original papers for the meanings of other parameters.

For CoF, we searched for the value of $H$ over the set $\{2,3,4,5,10,20,\dots,100\}$, and for $l$ we considered all levels of the hierarchical model obtained by HLTA. The number $K$ of latent variables on a level of the model was automatically determined by HLTA.

5.3 Evaluation Metrics

Two standard evaluation metrics are used in our experiments, namely recall@$R$, and area under the curve (AUC). The evaluation metrics are briefly described below:

• •

Recall@$R$: It is the fraction, among all items consumed by a user in the test set, of those that are placed at the top $R$ positions in a recommended list. Formally, recall is defined as: $\frac{TP}{TP+FN}$, where $TP$ denotes true positive and $FN$ denotes false negative.

• •

AUC : Is the area under the Receiver Operating Characteristic curve. It is the probability that two items randomly picked from the recommendation list are in the correct order, i.e., the first is a consumed item and the second is an unconsumed item.

5.4 Results

The recall@$R$ results, at each cutoff position $R$, are shown in Figure 3. We see that CoF achieved the best performance on both datasets. Since CoF attempts to model each taste of a user individually, a higher recall suggests that these tastes are catered for while making recommendations. The improvements are comparatively larger on the Movielens20M dataset. This indicates that when the data is relatively less sparse, CoF is able to extract meaningful information much more effectively than other methods.

Ocular performed competitively with BPR and WRMF in terms of recall@R. However, WRMF and BPRMF performed better at larger cutoff values. Interestingly, we found that the performance of GBPR was better than all baselines on the ta-feng dataset but was unimpressive on Movielens20M despite extensive parameter tuning. A possible reason for this could be the tendency of GBPR to focus on popular items888During our experiments we found that GBPR has low global diversity. These results are not reported in the interest of space.. Ta-feng is grocery dataset and certain common items are in every customer’s basket (e.g. bread), focusing on these may lead to a higher recall. On the other hand, the movie domain is comparatively more personalized and focusing on popular items might not cater to different tastes of a user.

Table 3 shows the performance in terms of AUC. CoF outperforms the baselines on both datasets. This suggests that CoF is able to identify the “true” negatives and it puts them lower than the items of interest in the ranked list. We see that BPRMF performs the second best w.r.t. AUC on both datasets. This is expected since BPRMF optimizes for AUC. Moreover, we note that WRMF performed better than BPRMF in terms of recall@$R$, however, it’s performance in terms of AUC is lower. This provides further evidence that the score of the top-$R$ metrics and those which evaluate over the whole list might not correlate and depending on the target of the recommendation an appropriate metric should be chosen.

SLIM gave the lowest performance over all metrics in our experiments. It is worth noting that our experimental setup (splitting the data by time) and the metrics used differ from the experimental conditions under which SLIM is normally evaluated.

5.5 Impact of Parameters

CoF begins by running HLTA to learn a hierarchical model and then uses the model for item recommendation. There are two parameters. The first parameter $l$ determines which level of the hierarchy to use. The larger the $l$, the fewer the number of user taste groups. The second parameter $H$ determines the amount of consumption history to use when characterizing user groups. Although both parameters are selected via validation, it would be interesting to gain some insights about how they impact the performance of CoF.

Figure 2(a) shows the recall@$20$ scores on ta-feng as a function of $H$ when $l=1$. We see that the curve first increases with $H$ and then decreases with $H$. It reaches the maximum value when $H=20$. The reason is that, when $H$ is too small, the data used for user taste group characterization contain too little information. When $H$ is too large, on the other hand, too much history is included and the data do not reflect the current interests of the user groups.

Figure 2(b) shows the AUC scores on ta-feng as a function of $H$ when $l=1$. We observe a similar trend, but the impacts of $H$ are not as pronounced on AUC. CoF is more or less robust to the choice of $l$ and the performance is almost the same regardless of the level chosen. The results are not shown for brevity. This was somewhat unexpected as when $l$ increases the taste become more general and one would expect the performance to deteriorate999We did observe slight deterioration but the magnitude was too small to draw conclusions from..

6 Conclusion

A novel method called CoF is proposed for collaborative filtering with implicit feedback data. It deals with the lack of negative examples which does not perform negative sampling, rather it uses the group UAN assumption, which is more reasonable than assumptions made by previous works. Extensive experiments were performed to compare CoF with a variety of baselines on two real-world datasets. CoF achieved the best performance over both recall@$R$ and AUC signifying that various taste of an individual is captured and the true negatives are placed at the bottom of the ranked list.

Acknowledgment

Research on this article was supported by Hong Kong Research Grants Council under grant 16202118.