KGAT: Knowledge Graph Attention Network for Recommendation

TL;DR

KGAT leverages knowledge graphs and attention mechanisms to model high-order relations for more accurate, diverse, and explainable recommendations, outperforming existing methods on benchmark datasets.

Contribution

Introduces KGAT, a novel end-to-end model that explicitly captures high-order relations in knowledge graphs for improved recommendation performance.

Findings

KGAT significantly outperforms Neural FM and RippleNet on benchmarks.

Embedding propagation effectively models high-order relations.

Attention mechanism enhances interpretability of recommendations.

Abstract

To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account. Traditional methods like factorization machine (FM) cast it as a supervised learning problem, which assumes each interaction as an independent instance with side information encoded. Due to the overlook of the relations among instances or items (e.g., the director of a movie is also an actor of another movie), these methods are insufficient to distill the collaborative signal from the collective behaviors of users. In this work, we investigate the utility of knowledge graph (KG), which breaks down the independent interaction assumption by linking items with their attributes. We argue that in such a hybrid structure of KG and user-item graph, high-order relations --- which connect two items with one or multiple linked attributes --- are an essential factor for successful recommendation. We propose a new method named Knowledge Graph Attention Network (KGAT) which explicitly models the high-order connectivities in KG in an end-to-end fashion. It recursively propagates the embeddings from a node's neighbors (which can be users, items, or attributes) to refine the node's embedding, and employs an attention mechanism to discriminate the importance of the neighbors. Our KGAT is conceptually advantageous to existing KG-based recommendation methods, which either exploit high-order relations by extracting paths or implicitly modeling them with regularization. Empirical results on three public benchmarks show that KGAT significantly outperforms state-of-the-art methods like Neural FM and RippleNet. Further studies verify the efficacy of embedding propagation for high-order relation modeling and the interpretability benefits brought by the attention mechanism.

Tables

Table 1. Table 1. Statistics of the datasets.

		Amazon-book	Last-FM	Yelp2018
User-Item Interaction	#Users	$70,679$	$23,566$	$45,919$
	#Items	$24,915$	$48,123$	$45,538$
	#Interactions	$847,733$	$3,034,796$	$1,185,068$
Knowledge Graph	#Entities	$88,572$	$58,266$	$90,961$
	#Relations	$39$	$9$	$42$
	#Triplets	$2,557,746$	$464,567$	$1,853,704$

Table 2. Table 2. Overall Performance Comparison.

	Amazon-Book		Last-FM		Yelp2018
	recall	ndcg	recall	ndcg	recall	ndcg
FM	$0.1345$	$0.0886$	$0.0778$	$0.1181$	$0.0627$	$0.0768$
NFM	$0.1366$	$0.0913$	$0.0829$	$0.1214$	$0.0660$	$0.0810$
CKE	$0.1343$	$0.0885$	$0.0736$	$0.1184$	$0.0657$	$0.0805$
CFKG	$0.1142$	$0.0770$	$0.0723$	$0.1143$	$0.0522$	$0.0644$
MCRec	$0.1113$	$0.0783$	-	-	-	-
RippleNet	$0.1336$	$0.0910$	$0.0791$	$0.1238$	$0.0664$	$0.0822$
GC-MC	$0.1316$	$0.0874$	$0.0818$	$0.1253$	$0.0659$	$0.0790$
KGAT	${0.1489}^{\ast }$	${0.1006}^{\ast }$	${0.0870}^{\ast }$	${0.1325}^{\ast }$	${0.0712}^{\ast }$	${0.0867}^{\ast }$
%Improv.	$8.95\%$	$10.05\%$	$4.93\%$	$5.77\%$	$7.18\%$	$5.54\%$

Table 3. Table 3. Effect of embedding propagation layer numbers ( L 𝐿 L ).

	Amazon-Book		Last-FM		Yelp2018
	recall	ndcg	recall	ndcg	recall	ndcg
KGAT-1	0.1393	0.0948	0.0834	0.1286	0.0693	0.0848
KGAT-2	0.1464	0.1002	0.0863	0.1318	0.0714	0.0872
KGAT-3	0.1489	0.1006	0.0870	0.1325	0.0712	0.0867
KGAT-4	0.1503	0.1015	0.0871	0.1329	0.0722	0.0871

Table 4. Table 4. Effect of aggregators.

	Amazon-Book		Last-FM		Yelp2018
Aggregator	recall	ndcg	recall	ndcg	recall	ndcg
GCN	0.1381	0.0931	0.0824	0.1278	0.0688	0.0847
GraphSage	0.1372	0.0929	0.0822	0.1268	0.0666	0.0831
Bi-Interaction	0.1393	0.0948	0.0834	0.1286	0.0693	0.0848

Table 5. Table 5. Effect of knowledge graph embedding and attention mechanism.

	Amazon-Book		Last-FM		Yelp2018
	recall	ndcg	recall	ndcg	recall	ndcg
w/o K&A	0.1367	0.0928	0.0819	0.1252	0.0654	0.0808
w/o KGE	0.1380	0.0933	0.0826	0.1273	0.0664	0.0824
w/o Att	0.1377	0.0930	0.0826	0.1270	0.0657	0.0815

Equations

$$g(h,r,t)={\Vert {\mathbf{W}}_r{\mathbf{e}}_h+{\mathbf{e}}_r-{\mathbf{W}}_r{\mathbf{e}}_t\Vert }_2^2,$$

$$\displaystyle\mathbf{\mathop{\mathcal{L}}}_{\text{KG}}=\sum_{(h,r,t,t^{\prime})\in\mathcal{T}}-\ln\sigma\Big{(}g(h,r,t^{\prime})-g(h,r,t)\Big{)},$$

$${\mathbf{e}}_{{\mathcal{N}}_h}=\sum _{(h,r,t)\in {\mathcal{N}}_h}\pi (h,r,t){\mathbf{e}}_t,$$

$$\displaystyle\pi(h,r,t)={(\mathbf{W}_{r}\mathbf{e}_{t})}^{\top}\text{tanh}\Big{(}(\mathbf{W}_{r}\mathbf{e}_{h}+\mathbf{e}_{r})\Big{)},$$

$$\pi (h,r,t)=\frac{\exp (\pi (h,r,t))}{{\sum }_{(h,r^{\prime },t^{\prime })\in {\mathcal{N}}_h}\exp (\pi (h,r^{\prime },t^{\prime }))}.$$

$$\displaystyle f_{\text{GCN}}=\text{LeakyReLU}\Big{(}\mathbf{W}(\mathbf{e}_{h}+\mathbf{e}_{\mathcal{N}_{h}})\Big{)},$$

$$\displaystyle f_{\text{GraphSage}}=\text{LeakyReLU}\Big{(}\mathbf{W}(\mathbf{e}_{h}||\mathbf{e}_{\mathcal{N}_{h}})\Big{)},$$

$$f_{\text{Bi-Interaction}}=$$

$$\displaystyle\text{LeakyReLU}\Big{(}\mathbf{W}_{2}(\mathbf{e}_{h}\odot\mathbf{e}_{\mathcal{N}_{h}})\Big{)},$$

$$\displaystyle\mathbf{e}_{h}^{(l)}=f\Big{(}\mathbf{e}^{(l-1)}_{h},\mathbf{e}^{(l-1)}_{\mathcal{N}_{h}}\Big{)},$$

$${\mathbf{e}}_{{\mathcal{N}}_h}^{(l-1)}=\sum _{(h,r,t)\in {\mathcal{N}}_h}\pi (h,r,t){\mathbf{e}}_t^{(l-1)},$$

$${\mathbf{e}}_u^{\ast }={\mathbf{e}}_u^{(0)}\Vert \cdots \Vert {\mathbf{e}}_u^{(L)},{\mathbf{e}}_i^{\ast }={\mathbf{e}}_i^{(0)}\Vert \cdots \Vert {\mathbf{e}}_i^{(L)},$$

$$\hat{y}(u,i)={{\mathbf{e}}_u^{\ast }}^{\top }{\mathbf{e}}_i^{\ast }.$$

$$\displaystyle\mathbf{\mathop{\mathcal{L}}}_{\text{CF}}=\sum_{(u,i,j)\in\mathcal{O}}-\ln\sigma\Big{(}\hat{y}(u,i)-\hat{y}(u,j)\Big{)}$$

$${\mathcal{L}}_{\text{KGAT}}={\mathcal{L}}_{\text{KG}}+{\mathcal{L}}_{\text{CF}}+\lambda {\Vert \Theta \Vert }_2^2,$$

Full text

In this work, we investigate the utility of knowledge graph (KG), which breaks down the independent interaction assumption by linking items with their attributes. We argue that in such a hybrid structure of KG and user-item graph, high-order relations — which connect two items with one or multiple linked attributes — are an essential factor for successful recommendation. We propose a new method named Knowledge Graph Attention Network (KGAT) which explicitly models the high-order connectivities in KG in an end-to-end fashion. It recursively propagates the embeddings from a node’s neighbors (which can be users, items, or attributes) to refine the node’s embedding, and employs an attention mechanism to discriminate the importance of the neighbors. Our KGAT is conceptually advantageous to existing KG-based recommendation methods, which either exploit high-order relations by extracting paths or implicitly modeling them with regularization. Empirical results on three public benchmarks show that KGAT significantly outperforms state-of-the-art methods like Neural FM (He and Chua, 2017) and RippleNet (Wang et al., 2018b). Further studies verify the efficacy of embedding propagation for high-order relation modeling and the interpretability benefits brought by the attention mechanism. We release the codes and datasets at https://github.com/xiangwang1223/knowledge_graph_attention_network.

Collaborative Filtering, Recommendation, Graph Neural Network, Higher-order Connectivity, Embedding Propagation, Knowledge Graph

††journalyear: 2019††copyright: acmcopyright††conference: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 4–8, 2019; Anchorage, AK, USA††booktitle: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’19), August 4–8, 2019, Anchorage, AK, USA††price: 15.00††doi: 10.1145/3292500.3330989††isbn: 978-1-4503-6201-6/19/08††ccs: Information systems Recommender systems

1. Introduction

The success of recommendation system makes it prevalent in Web applications, ranging from search engines, E-commerce, to social media sites and news portals — without exaggeration, almost every service that provides content to users is equipped with a recommendation system. To predict user preference from the key (and widely available) source of user behavior data, much research effort has been devoted to collaborative filtering (CF) (He et al., 2018, 2017; Wang et al., 2019a). Despite its effectiveness and universality, CF methods suffer from the inability of modeling side information (Wang et al., 2017, 2018a), such as item attributes, user profiles, and contexts, thus perform poorly in sparse situations where users and items have few interactions. To integrate such information, a common paradigm is to transform them into a generic feature vector, together with user ID and item ID, and feed them into a supervised learning (SL) model to predict the score. Such a SL paradigm for recommendation has been widely deployed in industry (Zhou et al., 2018; Shan et al., 2016; Cheng et al., 2016), and some representative models include factorization machine (FM) (Rendle et al., 2011), NFM (neural FM) (He and Chua, 2017), Wide&Deep (Cheng et al., 2016), and xDeepFM (Lian et al., 2018), etc.

Although these methods have provided strong performance, a deficiency is that they model each interaction as an independent data instance and do not consider their relations. This makes them insufficient to distill attribute-based collaborative signal from the collective behaviors of users. As shown in Figure 1, there is an interaction between user $u_{1}$ and movie $i_{1}$, which is directed by the person $e_{1}$. CF methods focus on the histories of similar users who also watched $i_{1}$, *i.e., *$u_{4}$ and $u_{5}$; while SL methods emphasize the similar items with the attribute $e_{1}$, *i.e., *$i_{2}$. Obviously, these two types of information not only are complementary for recommendation, but also form a high-order relationship between a target user and item together. However, existing SL methods fail to unify them and cannot take into account the high-order connectivity, such as the users in the yellow circle who watched other movies directed by the same person $e_{1}$, or the items in the grey circle that share other common relations with $e_{1}$.

To address the limitation of feature-based SL models, a solution is to take the graph of item side information, *aka. *knowledge graph111A KG is typically described as a heterogeneous network consisting of entity-relation-entity triplets, where the entity can be an item or an attribute. (Cao et al., 2018b, a), into account to construct the predictive model. We term the hybrid structure of knowledge graph and user-item graph as collaborative knowledge graph (CKG). As illustrated in Figure 1, the key to successful recommendation is to fully exploit the high-order relations in CKG, *e.g., *the long-range connectivities:

• •

$u_{1}\xrightarrow{r_{1}}i_{1}\xrightarrow{-r_{2}}e_{1}\xrightarrow{r_{2}}i_{2}\xrightarrow{-r_{1}}\{u_{2},u_{3}\},$

• •

$u_{1}\xrightarrow{r_{1}}i_{1}\xrightarrow{-r_{2}}e_{1}\xrightarrow{r_{3}}\{i_{3},i_{4}\},$

which represent the way to the yellow and grey circle, respectively. Nevertheless, to exploit such high-order information the challenges are non-negligible: 1) the nodes that have high-order relations with the target user increase dramatically with the order size, which imposes computational overload to the model, and 2) the high-order relations contribute unequally to a prediction, which requires the model to carefully weight (or select) them.

Several recent efforts have attempted to leverage the CKG structure for recommendation, which can be roughly categorized into two types, path-based (Yu et al., 2014; Zhao et al., 2017; Hu et al., 2018; Wang et al., 2019b; Sun et al., 2018; Wang et al., 2018b) and regularization-based (Zhang et al., 2016; Wang et al., 2019b; Huang et al., 2018; Cao et al., 2019):

• •

Path-based methods extract paths that carry the high-order information and feed them into predictive model. To handle the large number of paths between two nodes, they have either applied path selection algorithm to select prominent paths (Sun et al., 2018; Wang et al., 2019b), or defined meta-path patterns to constrain the paths (Yu et al., 2013; Hu et al., 2018). One issue with such two-stage methods is that the first stage of path selection has a large impact on the final performance, but it is not optimized for the recommendation objective. Moreover, defining effective meta-paths requires domain knowledge, which can be rather labor-intensive for complicated KG with diverse types of relations and entities, since many meta-paths have to be defined to retain model fidelity.

• •

Regularization-based methods devise additional loss terms that capture the KG structure to regularize the recommender model learning. For example, KTUP (Cao et al., 2019) and CFKG (Ai et al., 2018) jointly train the two tasks of recommendation and KG completion with shared item embeddings. Instead of directly plugging high-order relations into the model optimized for recommendation, these methods only encode them in an implicit manner. Due to the lack of an explicit modeling, neither the long-range connectivities are guaranteed to be captured, nor the results of high-order modeling are interpretable.

Considering the limitations of existing solutions, we believe it is of critical importance to develop a model that can exploit high-order information in KG in an efficient, explicit, and end-to-end manner. Towards this end, we take inspiration from the recent developments of graph neural networks (Kipf and Welling, 2017; Hamilton et al., 2017; Velickovic et al., 2018), which have the potential of achieving the goal but have not been explored much for KG-based recommendation. Specifically, we propose a new method named Knowledge Graph Attention Network (KGAT), which is equipped with two designs to correspondingly address the challenges in high-order relation modeling:

1. recursive embedding propagation, which updates a node’s embedding based on the embeddings of its neighbors, and recursively performs such embedding propagation to capture high-order connectivities in a linear time complexity; and
2. attention-based aggregation, which employs the neural attention mechanism (Vaswani et al., 2017; Chen et al., 2017) to learn the weight of each neighbor during a propagation, such that the attention weights of cascaded propagations can reveal the importance of a high-order connectivity. Our KGAT is conceptually advantageous to existing methods in that: 1) compared with path-based methods, it avoids the labor-intensive process of materializing paths, thus is more efficient and convenient to use, and 2) compared with regularization-based methods, it directly factors high-order relations into the predictive model, thus all related parameters are tailored for optimizing the recommendation objective.

The contributions of this work are summarized as follows:

• •

We highlight the importance of explicitly modeling the high-order relations in collaborative knowledge graph to provide better recommendation with item side information.

• •

We develop a new method KGAT, which achieves high-order relation modeling in an explicit and end-to-end manner under the graph neural network framework.

• •

We conduct extensive experiments on three public benchmarks, demonstrating the effectiveness of KGAT and its interpretability in understanding the importance of high-order relations.

2. Task Formulation

We first introduce the concept of CKG and highlight the high-order connectivity among nodes, as well as the compositional relations.

User-Item Bipartite Graph: In a recommendation scenario, we typically have historical user-item interactions (*e.g., *purchases and clicks). Here we represent interaction data as a user-item bipartite graph $\mathcal{G}_{1}$, which is defined as $\{(u,y_{ui},i)|u\in\mathcal{U},i\in\mathcal{I})\}$, where $\mathcal{U}$ and $\mathcal{I}$ separately denote the user and item sets, and a link $y_{ui}=1$ indicates that there is an observed interaction between user $u$ and item $i$; otherwise $y_{ui}=0$.

Knowledge Graph. In addition to the interactions, we have side information for items (*e.g., *item attributes and external knowledge). Typically, such auxiliary data consists of real-world entities and relationships among them to profile an item. For example, a movie can be described by its director, cast, and genres. We organize the side information in the form of knowledge graph $\mathcal{G}_{2}$, which is a directed graph composed of subject-property-object triple facts (Cao et al., 2019). Formally, it is presented as $\{(h,r,t)|h,t\in\mathcal{E},r\in\mathcal{R}\}$, where each triplet describes that there is a relationship $r$ from head entity $h$ to tail entity $t$. For example, (Hugh Jackman, ActorOf, Logan) states the fact that Hugh Jackman is an actor of the movie Logan. Note that $\mathcal{R}$ contains relations in both canonical direction (*e.g., *ActorOf) and inverse direction (*e.g., *ActedBy). Moreover, we establish a set of item-entity alignments $\mathcal{A}=\{(i,e)|i\in\mathcal{I},e\in\mathcal{E}\}$, where $(i,e)$ indicates that item $i$ can be aligned with an entity $e$ in the KG.

Collaborative Knowledge Graph. Here we define the concept of CKG, which encodes user behaviors and item knowledge as a unified relational graph. We first represent each user behavior as a triplet, $(u,\emph{Interact},i)$, where $y_{ui}=1$ is represented as an additional relation Interact between user $u$ and item $i$. Then based on the item-entity alignment set, the user-item graph can be seamlessly integrated with KG as a unified graph $\mathcal{G}=\{(h,r,t)|h,t\in\mathcal{E}^{\prime},r\in\mathcal{R}^{\prime}\}$, where $\mathcal{E}^{\prime}=\mathcal{E}\cup\mathcal{U}$ and $\mathcal{R}^{\prime}=\mathcal{R}\cup\{\emph{Interact}\}$.

Task Description We now formulate the recommendation task to be addressed in this paper:

• •

Input: collaborative knowledge graph $\mathcal{G}$ that includes the user-item bipartite graph $\mathcal{G}_{1}$ and knowledge graph $\mathcal{G}_{2}$.

• •

Output: a prediction function that predicts the probability $\hat{y}_{ui}$ that user $u$ would adopt item $i$.

High-Order Connectivity. Exploiting high-order connectivity is of importance to perform high-quality recommendation. Formally, we define the $L$-order connectivity between nodes as a multi-hop relation path: $e_{0}\xrightarrow{r_{1}}e_{1}\xrightarrow{r_{2}}\cdots\xrightarrow{r_{L}}e_{L}$, where $e_{l}\in\mathcal{E}^{\prime}$ and $r_{l}\in\mathcal{R}^{\prime}$; $(e_{l-1},r_{l},e_{l})$ is the $l$-th triplet, and $L$ is the length of the sequence. To infer user preference, CF methods build upon behavior similarity among users — more specifically similar users would exhibit similar preferences on items. Such intuition can be represented as behavior-based connectivity like $u_{1}\xrightarrow{r_{1}}i_{1}\xrightarrow{-r_{1}}u_{2}\xrightarrow{r_{1}}i_{2}$, which suggests that $u_{1}$ would exhibit preference on $i_{2}$, since her similar user $u_{2}$ has adopted $i_{2}$ before. Distinct from CF methods, SL models like FM and NFM focus on attributed-based connectivity, assuming that users tend to adopt items that share similar properties. For example, $u_{1}\xrightarrow{r_{1}}i_{1}\xrightarrow{r_{2}}e_{1}\xrightarrow{-r_{2}}i_{2}$ suggests that $u_{1}$ would adopt $i_{2}$ since it has the same director $e_{1}$ with $i_{1}$ she liked before. However, FM and NFM treat entities as the values of individual feature fields, failing to reveal relatedness across fields and related instances. For instance, it is hard to model $u_{1}\xrightarrow{r_{1}}i_{1}\xrightarrow{r_{2}}e_{1}\xrightarrow{-r_{3}}i_{2}$, although $e_{1}$ serves as the bridge connecting director and actor fields. We therefore argue that these methods do not fully explore the high-order connectivity and leave compositional high-order relations untouched.

3. Methodology

We now present the proposed KGAT model, which exploits high-order relations in an end-to-end fashion. Figure 2 shows the model framework, which consists of three main components: 1) embedding layer, which parameterizes each node as a vector by preserving the structure of CKG; 2) attentive embedding propagation layers, which recursively propagate embeddings from a node’s neighbors to update its representation, and employ knowledge-aware attention mechanism to learn the weight of each neighbor during a propagation; and 3) prediction layer, which aggregates the representations of a user and an item from all propagation layers, and outputs the predicted matching score.

3.1. Embedding Layer

Knowledge graph embedding is an effective way to parameterize entities and relations as vector representations, while preserving the graph structure. Here we employ TransR (Lin et al., 2015), a widely used method, on CKG. To be more specific, it learns embeds each entity and relation by optimizing the translation principle $\mathbf{e}^{r}_{h}+\mathbf{e}_{r}\approx\mathbf{e}^{r}_{t}$, if a triplet $(h,r,t)$ exists in the graph. Herein, $\mathbf{e}_{h},\mathbf{e}_{t}\in\mathbb{R}^{d}$ and $\mathbf{e}_{r}\in\mathbb{R}^{k}$ are the embedding for $h$, $t$, and $r$, respectively; and $\mathbf{e}_{h}^{r},\mathbf{e}_{t}^{r}$ are the projected representations of $\mathbf{e}_{h}$ and $\mathbf{e}_{t}$ in the relation $r$’s space. Hence, for a given triplet $(h,r,t)$, its plausibility score (or energy score) is formulated as follows:

[TABLE]

where $\mathbf{W}_{r}\in\mathbb{R}^{k\times d}$ is the transformation matrix of relation $r$, which projects entities from the $d$-dimension entity space into the $k$-dimension relation space. A lower score of $g(h,r,t)$ suggests that the triplet is more likely to be true true, and vice versa.

The training of TransR considers the relative order between valid triplets and broken ones, and encourages their discrimination through a pairwise ranking loss:

[TABLE]

where $\mathcal{T}=\{(h,r,t,t^{\prime})|(h,r,t)\in\mathcal{G},(h,r,t^{\prime})\not\in\mathcal{G}\}$, and $(h,r,t^{\prime})$ is a broken triplet constructed by replacing one entity in a valid triplet randomly; $\sigma(\cdot)$ is the sigmoid function. This layer models the entities and relations on the granularity of triples, working as a regularizer and injecting the direct connections into representations, and thus increases the model representation ability (evidences in Section 4.4.3.)

3.2. Attentive Embedding Propagation Layers

Next we build upon the architecture of graph convolution network (Kipf and Welling, 2017) to recursively propagate embeddings along high-order connectivity; moreover, by exploiting the idea of graph attention network (Velickovic et al., 2018), we generate attentive weights of cascaded propagations to reveal the importance of such connectivity. Here we start by describing a single layer, which consists of three components: information propagation, knowledge-aware attention, and information aggregation, and then discuss how to generalize it to multiple layers.

Information Propagation: One entity can be involved in multiple triplets, serving as the bridge connecting two triplets and propagating information. Taking $e_{1}\xrightarrow{r_{2}}i_{2}\xrightarrow{-r_{1}}u_{2}$ and $e_{2}\xrightarrow{r_{3}}i_{2}\xrightarrow{-r_{1}}u_{2}$ as an example, item $i_{2}$ takes attributes $e_{1}$ and $e_{2}$ as inputs to enrich its own features, and then contributes user $u_{2}$’s preferences, which can be simulated by propagating information from $e_{1}$ to $u_{2}$. We build upon this intuition to perform information propagation between an entity and its neighbors.

Considering an entity $h$, we use $\mathcal{N}_{h}=\{(h,r,t)|(h,r,t)\in\mathcal{G}\}$ to denote the set of triplets where $h$ is the head entity, termed ego-network (Qiu et al., 2018). To characterize the first-order connectivity structure of entity $h$, we compute the linear combination of $h$’s ego-network:

[TABLE]

where $\pi(h,r,t)$ controls the decay factor on each propagation on edge $(h,r,t)$, indicating how much information being propagated from $t$ to $h$ conditioned to relation $r$.

Knowledge-aware Attention: We implement $\pi(h,r,t)$ via relational attention mechanism, which is formulated as follows:

[TABLE]

where we select tanh (Velickovic et al., 2018) as the nonlinear activation function. This makes the attention score dependent on the distance between $e_{h}$ and $e_{t}$ in the relation $r$’s space, *e.g., *propagating more information for closer entities. Note that, we employ only inner product on these representations for simplicity, and leave the further exploration of the attention module as the future work.

Hereafter, we normalize the coefficients across all triplets connected with $h$ by adopting the softmax function:

[TABLE]

As a result, the final attention score is capable of suggesting which neighbor nodes should be given more attention to capture collaborative signals. When performing propagation forward, the attention flow suggests parts of the data to focus on, which can be treated as explanations behind the recommendation.

Distinct from the information propagation in GCN (Kipf and Welling, 2017) and GraphSage (Hamilton et al., 2017) which set the discount factor between two nodes as $1/\sqrt{|\mathcal{N}_{h}||\mathcal{N}_{t}|}$ or $1/|\mathcal{N}_{t}|$, our model not only exploits the proximity structure of graph, but also specify varying importance of neighbors. Moreover, distinct from graph attention network (Velickovic et al., 2018) which only takes node representations as inputs, we model the relation $e_{r}$ between $e_{h}$ and $e_{t}$, encoding more information during propagation. We perform experiments to verify the effectiveness of the attention mechanism and visualize the attention flow in Section 4.4.3 and Section 4.5, respectively.

Information Aggregation: The final phase is to aggregate the entity representation $\mathbf{e}_{h}$ and its ego-network representations $\mathbf{e}_{\mathcal{N}_{h}}$ as the new representation of entity $h$ — more formally, $\mathbf{e}_{h}^{(1)}=f(\mathbf{e}_{h},\mathbf{e}_{\mathcal{N}_{h}})$. We implement $f(\cdot)$ using three types of aggregators:

• •

GCN Aggregator (Kipf and Welling, 2017) sums two representations up and applies a nonlinear transformation, as follows:

[TABLE]

where we set the activation function set as LeakyReLU (Maas et al., 2013); $\mathbf{W}\in\mathbb{R}^{d^{\prime}\times d}$ are the trainable weight matrices to distill useful information for propagation, and $d^{\prime}$ is the transformation size.

• •

GraphSage Aggregator (Hamilton et al., 2017) concatenates two representations, followed by a nonlinear transformation:

[TABLE]

where $||$ is the concatenation operation.

• •

Bi-Interaction Aggregator is carefully designed by us to consider two kinds of feature interactions between $\mathbf{e}_{h}$ and $\mathbf{e}_{\mathcal{N}_{h}}$, as follows:

[TABLE]

where $\mathbf{W}_{1},\mathbf{W}_{2}\in\mathbb{R}^{d^{\prime}\times d}$ are the trainable weight matrices, and $\odot$ denotes the element-wise product. Distinct from GCN and GraphSage aggregators, we additionally encode the feature interaction between $\mathbf{e}_{h}$ and $\mathbf{e}_{\mathcal{N}_{h}}$. This term makes the information being propagated sensitive to the affinity between $\mathbf{e}_{h}$ and $\mathbf{e}_{\mathcal{N}_{h}}$, *e.g., *passing more messages from similar entities.

To summarize, the advantage of the embedding propagation layer lies in explicitly exploiting the first-order connectivity information to relate user, item, and knowledge entity representations. We empirically compare the three aggregators in Section 4.4.2.

High-order Propagation: We can further stack more propagation layers to explore the high-order connectivity information, gathering the information propagated from the higher-hop neighbors. More formally, in the $l$-th steps, we recursively formulate the representation of an entity as:

[TABLE]

wherein the information propagated within $l$-ego network for the entity $h$ is defined as follows,

[TABLE]

$\mathbf{e}_{t}^{(l-1)}$ is the representation of entity $t$ generated from the previous information propagation steps, memorizing the information from its $(l-1)$-hop neighbors; $\mathbf{e}_{h}^{(0)}$ is set as $\mathbf{e}_{h}$ at the initial information-propagation iteration. It further contributes to the representation of entity $h$ at layer $l$. As a result, high-order connectivity like $u_{2}\xrightarrow{r_{1}}i_{2}\xrightarrow{-r_{2}}e_{1}\xrightarrow{r_{2}}i_{1}\xrightarrow{-r_{1}}u_{1}$ can be captured in the embedding propagation process. Furthermore, the information from $u_{2}$ is explicitly encoded in $\mathbf{e}^{(3)}_{u_{1}}$. Clearly, the high-order embedding propagation seamlessly injects the attribute-based collaborative signal into the representation learning process.

3.3. Model Prediction

After performing $L$ layers, we obtain multiple representations for user node $u$, namely $\{\mathbf{e}^{(1)}_{u},\cdots,\mathbf{e}^{(L)}_{u}\}$; analogous to item node $i$, $\{\mathbf{e}^{(1)}_{i},\cdots,\mathbf{e}^{(L)}_{i}\}$ are obtained. As the output of the $l$-th layer is the message aggregation of the tree structure depth of $l$ rooted at $u$ (or $i$) as shown in Figure 1, the outputs in different layers emphasize the connectivity information of different orders. We hence adopt the layer-aggregation mechanism (Xu et al., 2018) to concatenate the representations at each step into a single vector, as follows:

[TABLE]

where $\|$ is the concatenation operation. By doing so, we not only enrich the initial embeddings by performing the embedding propagation operations, but also allow controlling the strength of propagation by adjusting $L$.

Finally, we conduct inner product of user and item representations, so as to predict their matching score:

[TABLE]

3.4. Optimization

To optimize the recommendation model, we opt for the BPR loss (Rendle et al., 2009). Specifically, it assumes that the observed interactions, which indicate more user preferences, should be assigned higher prediction values than unobserved ones:

[TABLE]

where $\mathcal{O}=\{(u,i,j)|(u,i)\in\mathcal{R}^{+},(u,j)\in\mathcal{R}^{-}\}$ denotes the training set, $\mathcal{R}^{+}$ indicates the observed (positive) interactions between user $u$ and item $j$ while $\mathcal{R}^{-}$ is the sampled unobserved (negative) interaction set; $\sigma(\cdot)$ is the sigmoid function.

Finally, we have the objective function to learn Equations (2) and (13) jointly, as follows:

[TABLE]

where $\Theta=\{\mathbf{E},\mathbf{W}_{r},\forall l\in\mathcal{R},\mathbf{W}_{1}^{(l)},\mathbf{W}_{2}^{(l)},\forall l\in\{1,\cdots,L\}\}$ is the model parameter set, and $\mathbf{E}$ is the embedding table for all entities and relations; $L_{2}$ regularization parameterized by $\lambda$ on $\Theta$ is conducted to prevent overfitting. It is worth pointing out that in terms of model size, the majority of model parameters comes from the entity embeddings (*e.g., *6.5 million on experimented Amazon dataset), which is almost identical to that of FM; the propagation layer weights are lightweight (*e.g., *5.4 thousand for the tower structure of three layers, *i.e., *$64-32-16-8$, on the Amazon dataset).

3.4.1. Training:

We optimize $\mathbf{\mathop{\mathcal{L}}}_{KG}$ and $\mathbf{\mathop{\mathcal{L}}}_{CF}$ alternatively, where mini-batch Adam (Kingma and Ba, 2014) is adopted to optimize the embedding loss and the prediction loss. Adam is a widely used optimizer, which is able to adaptively control the learning rate *w.r.t. *the absolute value of gradient. In particular, for a batch of randomly sampled $(h,r,t,t^{\prime})$, we update the embeddings for all nodes; hereafter, we sample a batch of $(u,i,j)$ randomly, retrieve their representations after $L$ steps of propagation, and then update model parameters by using the gradients of the prediction loss.

3.4.2. Time Complexity Analysis:

As we adopt the alternative optimization strategy, the time cost mainly comes from two parts. For the knowledge graph embedding (*cf. *Equation (2)), the translation principle has computational complexity $O(|\mathcal{G}_{2}|d^{2})$. For the attention embedding propagation part, the matrix multiplication of the $l$-th layer has computational complexity $O(|\mathcal{G}|d_{l}d_{l-1})$; and $d_{l}$ and $d_{l-1}$ are the current and previous transformation size. For the final prediction layer, only the inner product is conducted, for which the time cost of the whole training epoch is $O(\sum_{l=1}^{L}|\mathcal{G}|d_{l})$. Finally, the overall training complexity of KGAT is $O(|\mathcal{G}_{2}|d^{2}+\sum_{l=1}^{L}|\mathcal{G}|d_{l}d_{l-1}+|\mathcal{G}|d_{l})$.

As online services usually require real-time recommendation, the computational cost during inference is more important that that of training phase. Empirically, FM, NFM, CFKG, CKE, GC-MC, KGAT, MCRec, and RippleNet cost around $700$s, $780$s, $800$s, $420$s, $500$s, $\mathbf{560}$s, $20$ hours, and $2$ hours for all testing instances on Amazon-Book dataset, respectively. As we can see, KGAT achieves comparable computation complexity to SL models (FM and NFM) and regularization-based methods (CFKG and CKE), being much efficient that path-based methods (MCRec and RippleNet).

4. Experiments

We evaluate our proposed method, especially the embedding propagation layer, on three real-world datasets. We aim to answer the following research questions:

• •

RQ1: How does KGAT perform compared with state-of-the-art knowledge-aware recommendation methods?

• •

RQ2: How do different components (*i.e., *knowledge graph embedding, attention mechanism, and aggregator selection) affect KGAT?

• •

RQ3: Can KGAT provide reasonable explanations about user preferences towards items?

4.1. Dataset Description

To evaluate the effectiveness of KGAT, we utilize three benchmark datasets: Amazon-book, Last-FM, and Yelp2018, which are publicly accessible and vary in terms of domain, size, and sparsity.

Amazon-book222http://jmcauley.ucsd.edu/data/amazon.: Amazon-review is a widely used dataset for product recommendation (He and McAuley, 2016). We select Amazon-book from this collection. To ensure the quality of the dataset, we use the $10$-core setting, *i.e., *retaining users and items with at least ten interactions.

Last-FM333https://grouplens.org/datasets/hetrec-2011/.: This is the music listening dataset collected from Last.fm online music systems. Wherein, the tracks are viewed as the items. In particular, we take the subset of the dataset where the timestamp is from Jan, 2015 to June, 2015. We use the same $10$-core setting in order to ensure data quality.

Yelp2018444https://www.yelp.com/dataset/challenge.: This dataset is adopted from the 2018 edition of the Yelp challenge. Here we view the local businesses like restaurants and bars as the items. Similarly, we use the $10$-core setting to ensure that each user and item have at least ten interactions.

Besides the user-item interactions, we need to construct item knowledge for each dataset. For Amazon-book and Last-FM, we follow the way in (Zhao et al., 2018) to map items into Freebase entities via title matching if there is a mapping available. In particular, we consider the triplets that are directly related to the entities aligned with items, no matter which role (*i.e., *subject or object) it serves as. Distinct from existing knowledge-aware datasets that provide only one-hop entities of items, we also take the triplets that involve two-hop neighbor entities of items into consideration. For Yelp2018, we extract item knowledge from the local business information network (*e.g., *category, location, and attribute) as KG data. To ensure the KG quality, we then preprocess the three KG parts by filtering out infrequent entities (*i.e., *lowever than $10$ in both datasets) and retaining the relations appearing in at least $50$ triplets. We summarize the statistics of three datasets in Table 1.

For each dataset, we randomly select $80\%$ of interaction history of each user to constitute the training set, and treat the remaining as the test set. From the training set, we randomly select $10\%$ of interactions as validation set to tune hyper-parameters. For each observed user-item interaction, we treat it as a positive instance, and then conduct the negative sampling strategy to pair it with one negative item that the user did not consume before.

4.2. Experimental Settings

4.2.1. Evaluation Metrics

For each user in the test set, we treat all the items that the user has not interacted with as the negative items. Then each method outputs the user’s preference scores over all the items, except the positive ones in the training set. To evaluate the effectiveness of top-$K$ recommendation and preference ranking, we adopt two widely-used evaluation protocols (He et al., 2017; Yang et al., 2018): recall@$K$ and ndcg@$K$. By default, we set $K=20$. We report the average metrics for all users in the test set.

4.2.2. Baselines

To demonstrate the effectiveness, we compare our proposed KGAT with SL (FM and NFM), regularization-based (CFKG and CKE), path-based (MCRec and RippleNet), and graph neural network-based (GC-MC) methods, as follows:

• •

FM (Rendle et al., 2011): This is a bechmark factorization model, where considers the second-order feature interactions between inputs. Here we treat IDs of a user, an item, and its knowledge (*i.e., *entities connected to it) as input features.

• •

NFM (He and Chua, 2017): The method is a state-of-the-art factorization model, which subsumes FM under neural network. Specially, we employed one hidden layer on input features as suggested in (He and Chua, 2017).

• •

CKE (Zhang et al., 2016): This is a representative regularization-based method, which exploits semantic embeddings derived from TransR (Lin et al., 2015) to enhance matrix factorization (Rendle et al., 2009).

• •

CFKG (Ai et al., 2018): The model applies TransE (Bordes et al., 2013) on the unified graph including users, items, entities, and relations, casting the recommendation task as the plausibility prediction of $(u,\emph{Interact},i)$ triplets.

• •

MCRec (Hu et al., 2018): This is a path-based model, which extracts qualified meta-paths as connectivity between a user and an item.

• •

RippleNet (Wang et al., 2018b): Such model combines regularization- and path-based methods, which enrich user representations by adding that of items within paths rooted at each user.

• •

GC-MC (van den Berg et al., 2017): Such model is designed to employ GCN (Kipf and Welling, 2017) encoder on graph-structured data, especially for the user-item bipartite graph. Here we apply it on the user-item knowledge graph. Especially, we employ one graph convolution layers as suggested in (van den Berg et al., 2017), where the hidden dimension is set equal to the embedding size.

4.2.3. Parameter Settings

We implement our KGAT model in Tensorflow. The embedding size is fixed to $64$ for all models, except RippleNet $16$ due to its high computational cost. We optimize all models with Adam optimizer, where the batch size is fixed at $1024$. The default Xavier initializer (Glorot and Bengio, 2010) to initialize the model parameters. We apply a grid search for hyper-parameters: the learning rate is tuned amongst $\{0.05,0.01,0.005,0.001\}$, the coefficient of $L_{2}$ normalization is searched in $\{10^{-5},10^{-4},\cdots,10^{1},10^{2}\}$, and the dropout ratio is tuned in $\{0.0,0.1,\cdots,0.8\}$ for NFM, GC-MC, and KGAT. Besides, we employ the node dropout technique for GC-MC and KGAT, where the ratio is searched in $\{0.0,0.1,\cdots,0.8\}$. For MCRec, we manually define several types of user-item-attribute-item meta-paths, such as user-book-author-user and user-book-genre-user for Amazon-book dataset; we set the hidden layers as suggested in (Hu et al., 2018), which is a tower structure with $512$, $256$, $128$, $64$ dimensions. For RippleNet, we set the number of hops and the memory size as $2$ and $8$, respectively. Moreover, early stopping strategy is performed, *i.e., *premature stopping if recall@$20$ on the validation set does not increase for $50$ successive epochs. To model the third-order connectivity, we set the depth of KGAT $L$ as three with hidden dimension $64$, $32$, and $16$, respectively; we also report the effect of layer depth in Section 4.4.1. For each layer, we conduct the Bi-Interaction aggregator.

4.3. Performance Comparison (RQ1)

We first report the performance of all the methods, and then investigate how the modeling of high-order connectivity alleviate the sparsity issues.

4.3.1. Overall Comparison

The performance comparison results are presented in Table 2. We have the following observations:

• •

KGAT consistently yields the best performance on all the datasets. In particular, KGAT improves over the strongest baselines *w.r.t. *recall@$20$ by $8.95\%$, $4.93\%$, and $7.18\%$ in Amazon-book, Last-FM, and Yelp2018, respectively. By stacking multiple attentive embedding propagation layers, KGAT is capable of exploring the high-order connectivity in an explicit way, so as to capture collaborative signal effectively. This verifies the significance of capturing collaborative signal to transfer knowledge. Moreover, compared with GC-MC, KGAT justifies the effectiveness of the attention mechanism, specifying the attentive weights *w.r.t. *compositional semantic relations, rather than the fixed weights used in GC-MC.

• •

SL methods (*i.e., *FM and NFM) achieve better performance than the CFKG and CKE in most cases, indicating that regularization-based methods might not make full use of item knowledge. In particular, to enrich the representation of an item, FM and NFM exploit the embeddings of its connected entities, while CFKG and CKE only use that of its aligned entities. Furthermore, the cross features in FM and NFM actually serve as the second-order connectivity between users and entities, whereas CFKG and CKE model connectivity on the granularity of triples, leaving high-order connectivity untouched.

• •

Compared to FM, the performance of RippleNet verifies that incorporating two-hop neighboring items is of importance to enrich user representations. It therefore points to the positive effect of modeling the high-order connectivity or neighbors. However, RippleNet slightly underperforms NFM in Amazon-book and Last-FM, while performing better in Yelp2018. One possible reason is that NFM has stronger expressiveness, since the hidden layer allows NFM to capture the nonlinear and complex feature interactions between user, item, and entity embeddings.

• •

RippleNet outperforms MCRec by a large margin in Amazon-book. One possible reason is that MCRec depends heavily on the quality of meta-paths, which require extensive domain knowledge to define. The observation is consist with (Wang et al., 2018b).

• •

GC-MC achieves comparable performance to RippleNet in Last-FM and Yelp2018 datasets. While introducing the high-order connectivity into user and item representations, GC-MC forgoes the semantic relations between nodes; whereas RippleNet utilizes relations to guide the exploration of user preferences.

4.3.2. **Performance Comparison w.r.t. Interaction Sparsity Levels

One motivation to exploiting KG is to alleviate the sparsity issue, which usually limits the expressiveness of recommender systems. It is hard to establish optimal representations for inactive users with few interactions. Here we investigate whether exploiting connectivity information helps alleviate this issue.

Towards this end, we perform experiments over user groups of different sparsity levels. In particular, we divide the test set into four groups based on interaction number per user, meanwhile try to keep different groups have the same total interactions. Taking Amazon-book dataset as an example, the interaction numbers per user are less than $7$, $15$, $48$, and $4475$ respectively. Figure 3 illustrates the results *w.r.t. *ndcg@$20$ on different user groups in Amazon-book, Last-FM, and Yelp2018. We can see that:

• •

KGAT outperforms the other models in most cases, especially on the two sparsest user groups in Amazon-Book and Yelp2018. It again verifies the significance of high-order connectivity modeling, which 1) contains the lower-order connectivity used in baselines, and 2) enriches the representations of inactive users via recursive embedding propagation.

• •

It is worthwhile pointing out that KGAT slightly outperforms some baselines in the densest user group (*e.g., *the $<2057$ group of Yelp2018). One possible reason is that the preferences of users with too many interactions are too general to capture. High-order connectivity could introduce more noise into the user preferences, thus leading to the negative effect.

4.4. Study of KGAT (RQ2)

To get deep insights on the attentive embedding propagation layer of KGAT, we investigate its impact. We first study the influence of layer numbers. In what follows, we explore how different aggregators affect the performance. We then examine the influence of knowledge graph embedding and attention mechanism.

4.4.1. Effect of Model Depth

We vary the depth of KGAT (*e.g., *$L$) to investigate the efficiency of usage of multiple embedding propagation layers. In particular, the layer number is searched in the range of $\{1,2,3,4\}$; we use KGAT-1 to indicate the model with one layer, and similar notations for others. We summarize the results in Table 3, and have the following observations:

• •

Increasing the depth of KGAT is capable of boosting the performance substantially. Clearly, KGAT-2 and KGAT-3 achieve consistent improvement over KGAT-1 across all the board. We attribute the improvements to the effective modeling of high-order relation between users, items, and entities, which is carried by the second- and third-order connectivities, respectively.

• •

Further stacking one more layer over KGAT-3, we observe that KGAT-4 only achieve marginal improvements. It suggests that considering third-order relations among entities could be sufficient to capture the collaborative signal, which is consistent to the findings in (Wang et al., 2019b; Hu et al., 2018).

• •

Jointly analyzing Tables 2 and 3, KGAT-1 consistently outperforms other baselines in most cases. It again verifies the effectiveness of that attentive embedding propagation, empirically showing that it models the first-order relation better.

4.4.2. Effect of Aggregators

To explore the impact of aggregators, we consider the variants of KGAT-1 that uses different settings — more specifically GCN, GraphSage, and Bi-Interaction (*cf. *Section  3.1), termed KGAT-1${}_{\text{GCN}}$, KGAT-1${}_{\text{GraphSage}}$, and KGAT-1${}_{\text{Bi}}$, respectively. Table 4 summarizes the experimental results. We have the following findings:

• •

KGAT-1${}_{\text{GCN}}$ is consistently superior to KGAT-1${}_{\text{GraphSage}}$. One possible reason is that GraphSage forgoes the interaction between the entity representation $\mathbf{e}_{h}$ and its ego-network representation $\mathbf{e}_{\mathcal{N}_{h}}$. It hence illustrates the importance of feature interaction when performing information aggregation and propagation.

• •

Compared to KGAT-1${}_{\text{GCN}}$, the performance of KGAT-1${}_{\text{Bi}}$ verifies that incorporating additional feature interaction can improve the representation learning. It again illustrates the rationality and effectiveness of Bi-Interaction aggregator.

4.4.3. Effect of Knowledge Graph Embedding and Attention Mechanism

To verify the impact of knowledge graph embedding and attention mechanism, we do ablation study by considering three variants of KGAT-1. In particular, we disable the TransR embedding component (*cf. *Equation (2)) of KGAT, termed KGAT-1${}_{\text{w/o KGE}}$; we disable the attention mechanism (*cf. *Equation (4)) and set $\pi(h,r,t)$ as $1/|\mathcal{N}_{h}|$, termed KGAT-1${}_{\text{w/o Att}}$. Moreover, we obtain another variant by removing both components, named KGAT-1${}_{\text{w/o K\&A}}$. We summarize the experimental results in Table 5 and have the following findings:

• •

Removing knowledge graph embedding and attention components degrades the model’s performance. KGAT-1${}_{\text{w/o K\&A}}$ consistently underperforms KGAT-1${}_{\text{w/o KGE}}$ and KGAT-1${}_{\text{w/o Att}}$. It makes sense since KGAT${}_{\text{w/o K\&A}}$ fails to explicitly model the representation relatedness on the granularity of triplets.

• •

Compared with KGAT-1${}_{\text{w/o Att}}$, KGAT-1${}_{\text{w/o KGE}}$ performs better in most cases. One possible reason is that treating all neighbors equally (*i.e., *KGAT-1${}_{\text{w/o Att}}$) might introduce noises and mislead the embedding propagation process. It verifies the substantial influence of graph attention mechanism.

4.5. Case Study (RQ3)

Benefiting from the attention mechanism, we can reason on high-order connectivity to infer the user preferences on the target item, offering explanations. Towards this end, we randomly selected one user $u_{208}$ from Amazon-Book, and one relevant item $i_{4293}$ (from the test, unseen in the training phase). We extract behavior-based and attribute-based high-order connectivity connecting the user-item pair, based on the attention scores. Figure 4 shows the visualization of high-order connectivity. There are two key observations:

• •

KGAT captures the behavior-based and attribute-based high-order connectivity, which play a key role to infer user preferences. The retrieved paths can be viewed as the evidence why the item meets the user’s preference. As we can see, the connectivity $u_{208}\xrightarrow{r_{0}}\emph{Old Man's War}\xrightarrow{r_{14}}\emph{John Scalzi}\xrightarrow{-r_{14}}i_{4293}$ has the highest attention score, labeled with the solid line in the left subfigure. Hence, we can generate the explanation as The Last Colony is recommended since you have watched Old Man’s War written by the same author John Scalzi.

• •

The quality of item knowledge is of crucial importance. As we can see, entity English with relation Original Language is involved in one path, which is too general to provide high-quality explanations. This inspires us to perform hard attention to filter less informative entities out in future work.

5. Conclusion and Future Work

In this work, we explore high-order connectivity with semantic relations in CKG for knowledge-aware recommendation. We devised a new framework KGAT, which explicitly models the high-order connectivities in CKG in an end-to-end fashion. At it core is the attentive embedding propagation layer, which adaptively propagates the embeddings from a node’s neighbors to update the node’s representation. Extensive experiments on three real-world datasets demonstrate the rationality and effectiveness of KGAT.

This work explores the potential of graph neural networks in recommendation, and represents an initial attempt to exploit structural knowledge with information propagation mechanism. Besides knowledge graph, many other structural information indeed exists in real-world scenarios, such as social networks and item contexts. For example, by integrating social network with CKG, we can investigate how social influence affects the recommendation. Another exciting direction is the integration of information propagation and decision process, which opens up research possibilities of explainable recommendation.

Acknowledgement: This research is part of NExT++ research and also supported by the Thousand Youth Talents Program 2018. NExT++ is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its IRC@SG Funding Initiative.