Triple-to-Text: Converting RDF Triples into High-Quality Natural Languages via Optimizing an Inverse KL Divergence

TL;DR

This paper introduces the Triple-to-Text framework that improves natural language generation from RDF triples by optimizing inverse KL divergence, leading to higher-quality sentences compared to traditional maximum likelihood methods.

Contribution

The paper proposes a novel T2T framework that directly optimizes inverse KL divergence, addressing the limitations of maximum likelihood estimation in RDF-to-text generation.

Findings

T2T outperforms baseline models on multiple datasets.

The method significantly reduces low-quality sentence generation.

Experimental results show improved evaluation metrics.

Abstract

Knowledge base is one of the main forms to represent information in a structured way. A knowledge base typically consists of Resource Description Frameworks (RDF) triples which describe the entities and their relations. Generating natural language description of the knowledge base is an important task in NLP, which has been formulated as a conditional language generation task and tackled using the sequence-to-sequence framework. Current works mostly train the language models by maximum likelihood estimation, which tends to generate lousy sentences. In this paper, we argue that such a problem of maximum likelihood estimation is intrinsic, which is generally irrevocable via changing network structures. Accordingly, we propose a novel Triple-to-Text (T2T) framework, which approximately optimizes the inverse Kullback-Leibler (KL) divergence between the distributions of the real and generated sentences. Due to the nature that inverse KL imposes large penalty on fake-looking samples, the proposed method can significantly reduce the probability of generating low-quality sentences. Our experiments on three real-world datasets demonstrate that T2T can generate higher-quality sentences and outperform baseline models in several evaluation metrics.

Tables

Table 1. Table 1. Glossary

Symbol	Description
$ℱ$	a knowledge base that consists of RDF triples
$t$	a resource description framework (RDF) triple
$⟨s_i,p_i,o_i⟩$	subject, predicate and object within a RDF triple
$S$	a sentence
$w$	a word in a sentence
$X$	conditional context for SEQ2SEQ framework
$Y$	target context for generative models
$x_i$	$i$-th token from conditional context
$y_i$	$i$-th token from target context
	prefix of target context: $\{y_1,y_2,\cdots ,y_{i-1}\}$
$P$	the target (ground-truth) distribution
$G_{\theta }$	learned distribution of generator
$M_{\varphi }$	learned distribution of judger
$\theta$	parameters of generator
$\varphi$	parameters of judger

Table 2. Table 2. Dataset statistics, including the number of RDF triples-sentence pairs used in training and test, the number of RDF triples per datum, the (maximum) number of tokens per sentence and the vocabulary list size.

Dataset	#Train	#Test	#RDF Triples	#sentence length	#vocabulary in sentence	#vocabulary in triples
WebNLG	20288	2240	1-7	82	4678	2718
SemEval	8000	2717	1	97	24986	7333
Baidu SKE	19520	2000	1-5	84	25027	22713

Table 3. Table 3. Comparison of model performance.

	BLEU-3 ↑			BLEU-4 ↑			TER ↓			METEOR ↑
	WebNLG	SemEval	SKE	WebNLG	SemEval	SKE	WebNLG	SemEval	SKE	WebNLG	SemEval	SKE
MLE	40.8	4.24	18.6	30.2	2.73	15.6	0.497	1.07	1.01	0.636	0.222	0.349
CoT	9.84	1.90	15.7	6.40	1.41	12.9	1.085	1.16	1.08	0.349	0.102	0.305
SeqGAN	42.0	4.11	19.0	24.4	2.63	14.1	0.534	1.11	1.10	0.597	0.231	0.344
PG	41.7	4.21	17.9	30.9	2.06	14.1	0.607	1.13	1.12	0.628	0.197	0.310
NW	35.8	2.80	14.6	24.6	1.87	11.9	1.664	1.17	1.92	0.302	0.143	0.301
T2T	42.4	4.35	20.3	32.2	2.83	17.1	0.473	0.957	0.947	0.641	0.247	0.367

Table 4. Table 4. predicate accuracy on SemEval dataset.

methods	MLE	CoT	PG	SeqGAN	NW	T2T
accuracy	0.240	0.258	0.231	0.244	0.155	0.276

Table 5. Table 5. Forward perplexity among three datasets.

methods	WebNLG	SemEval	SKE
MLE	1.810	2.918	7.046
CoT	2.423	2.579	6.643
SeqGAN	3.556	4.129	7.834
PG	2.151	3.180	8.078
NW	2.461	2.771	6.916
T2T	1.589	2.067	3.565

Table 6. Table 6. Human evaluation on WebNLG dataset.

methods	Grammar	Correctness
MLE	7.6	6.7
CoT	5.5	3.5
PG	7.1	5.6
SeqGAN	8.0	5.8
NW	6.3	4.6
T2T	8.6	7.1

Table 7. Table 7. Sample output of the system.

RDF inputs	¡Italy , capital , Rome¿, ¡Italy , leaderName , Matteo Renzi¿, ¡Amatriciana sauce , country , Italy¿,                                                         ¡Italy , leaderName , Laura Boldrini¿
Reference	Amatriciana sauce is a traditional sauce in italy ( the capital of which is rome ) , where two of the country ’ s leaders are matteo renzi and laura boldrini .
MLE	Italy is called a country Amatriciana sauce . Matteo Renzi and Laura Boldrini are leaders in Italy where the capital is Rome .
CoT	Laura Boldrini is a leader in Italy where Rome is the capital of the country of Italy where where valencia is bacon.
SeqGAN	Amatricana sauce comes from Italy , a political leader and the capital is Rome . matteo renzi and Laura Boldrini are one of the leaders of Italy is
PG	Amatriciana sauce , a traditional italian dish from the Rome of the italian , where Rome the the leader is either two leaders include Matteo Renzi.
NW	the leader of Italy is Laura Boldrini where amatriciana sauce can be found .
T2T	Amatriciana sauce is from the country of Italy where capital is Rome . its leader is Laura Boldrini and Matteo Renzi leads the country .

Equations

$${\mathbf{h}}_t^{\text{enc}}=f^{\text{enc}}({\mathbf{h}}_{t-1}^{\text{enc}},{\mathbf{e}}_{x_t}),$$

$${\mathbf{h}}_t^{\text{dec}}=f^{\text{dec}}({\mathbf{h}}_{t-1}^{\text{dec}},{\mathbf{e}}_{y_{t-1}},{\mathbf{c}}_t),$$

$$\mathrm{Pr}(y_t|x_1,x_2,\cdots ,x_L,y_{<t})=\mathrm{softmax}(W{h}_t^{\text{dec}}+b),$$

$$\mathrm{Pr}(Y|X)$$

$$=\prod _{t=1}^T\mathrm{Pr}(y_t|x_1,x_2,\cdots ,x_L,y_{<t}).$$

$$J_G(\theta )={\mathbb{E}}_{Y\sim P}[\log G_{\theta }(Y)]=-H(P,G_{\theta }),$$

$$\displaystyle\mathrm{KL}(P\,\|G_{\theta})=\mathbb{E}_{Y\sim P}\big{[}\log\dfrac{P(Y)}{G_{\theta}(Y)}\big{]}=H(P,G_{\theta})-H(P),$$

$$\mathrm{KL}(P\Vert G_{\theta })=\sum _{Y}P(Y)\log \frac{P(Y)}{G_{\theta }(Y)}.$$

$$\mathrm{KL}(G_{\theta }\Vert P)=H(G_{\theta },P)-H(G_{\theta }).$$

$$J_M(\varphi )={\mathbb{E}}_{(X,Y)\sim P}[\log M_{\varphi }(Y|X)].$$

$$\mathrm{JSD}(P\|G_{\theta})=\frac{1}{2}\big{(}\mathrm{KL}(P\|M)+\mathrm{KL}(G_{\theta}\|M)\big{)},$$

$$G_{\theta }(Y|X)$$

$$M_{\varphi }(Y|X)$$

$$J_M(\varphi )={\mathbb{E}}_{(X,Y)\sim P}[\log M_{\varphi }(Y|X)].$$

$$J_G(\theta )$$

$$\mathop{\mathbb{E}}_{(X,Y)\sim G_{\theta}}\Big{[}\mathop{\sum}_{t=1}^{T}\big{(}\mathop{\log}g_{\theta}(y_{t}|X,y_{<t})-\mathop{\log}m_{\phi}(y_{t}|X,y_{<t})\big{)}\Big{]}.$$

$$\hat{p}=\underset{p_i\in P}{\mathrm{argmax}}{G}_{\theta }(Y|X_{p_i})$$

$$\mathtt{FPPL}({\mathtt{G}}_{\theta })=e^{-{\mathbb{E}}_{y\sim G_{\theta }}\log H_{\psi }(y)}$$

Full text

Natural Language Generation, Sequence to Sequence Generation, Knowledge Bases

††journalyear: 2019††copyright: acmcopyright††conference: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval; July 21–25, 2019; Paris, France††booktitle: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’19), July 21–25, 2019, Paris, France††price: 15.00††doi: 10.1145/3331184.3331232††isbn: 978-1-4503-6172-9/19/07††ccs: Computing methodologies Natural language generation

1. Introduction

Knowledge bases (KB) are gaining attention for their wide range of industrial applications, including, question answering (Q&A) systems (Fader et al., 2014; Zou et al., 2014), search engines (Ding et al., 2004), recommender systems (Huang et al., 2002) etc. The Resource Description Frameworks (RDF) is the general framework for representing entities and their relations in a structured knowledge base. Based on W3C standard (Magazine, 1998), each RDF datum is a triple consisting of three elements, in the form of (subject, predicate, object). An instance can be found in Figure 1(a), which illustrates a knowledge base about Neil Armstrong and its corresponding RDF triples.

Based on the RDF triples, the Q&A systems can answer questions such as ”which country does Neil Armstrong come from?” Although such tuples in RDF allow machines to process knowledge efficiently, they are generally hard for humans to understand. Some human interaction interfaces (e.g., DBpedia111https://wiki.dbpedia.org/) are designed to deliver knowledge bases in the form of RDF triples in a human-readable way.

In this paper, given a knowledge base in the form of RDF triples, our goal is to generate natural language description of the knowledge bases which are grammatically correct, easy to understand, and capable of delivering the information to humans. Figure 1(b) lays out the natural language description given the knowlege base about Neil Armstrong.

Traditionally, the Triple-to-Text task relies on rules and templates (Dale et al., 2003; Turner et al., 2010; Cimiano et al., 2013), which requires a large number of human efforts. Moreover, even if these systems are developed, they are often faced with problems of low scalability and inability to handle complex logic.

Recently, with significant progress on deep learning, the neural network (NN) based natural language generation models, especially the sequence to sequence framework (SEQ2SEQ) (Sutskever et al., 2014), have achieved remarkable success in machine translation(Bahdanau et al., 2014) and text summarization(Nallapati et al., 2016). The SEQ2SEQ framework has also been employed to translate knowledge bases into natural languages. Vougiouklis et al. (Vougiouklis et al., 2018) proposed Neural Wikipedian to generate summaries of the RDF triples.

However, most existing studies focus on the design of the model structure (Vougiouklis et al., 2018), while paying less attention to the training objective. These models are usually trained via maximum likelihood estimation, which is equivalent to minimizing Kullback-Leibler (KL) divergence between the ground-truth conditional distribution ($P$) and the estimated distribution ($G$), i.e., $KL(P\|G)$. Models trained with KL divergence tend to have high diversity, but at the same time, they are likely to generate shoddy samples (Huszár, 2015).

In such tasks, we usually care more about the quality of the translation and care less about diversity. Hence, we propose the triple-to-text model. By introducing a new component called judger, we optimize the model in two directions: minimizing the approximated inverse KL divergence and maximizing the self-entropy.

Our main contributions can be summarized as follows:

• •

We propose a theoretically sound and empirically effective framework (T2T) for optimizing the inverse KL divergence for conditional language generation task of translating a knowledge base into its natural language description.

• •

We conduct a series of experiments on different datasets to validate our proposed method. The results show that our method outperforms baselines in common metrics.

We organize the remaining parts of this paper as follows. In Section 2, we formulate the problem and introduce the preliminaries. In Section 3, we provide our analysis of why it is preferable to optimize an inverse KL divergence. Then Section 4 details our proposed model. We then present the experiment results in Section 5. Finally, we discuss the related work in Section 6 and conclude the paper in Section 7.

2. Formulation and Preliminaries

In this section, we formulate the task and introduce the preliminaries of language generation models.

2.1. Task Definition

A knowledge base $\mathcal{F}$ is formulated as a set of RDF triples, i.e., $\mathcal{F}=\{t_{1},t_{2},\cdots,t_{N}\}$, where each RDF triple $t_{i}$ is represented as $\langle s_{i},p_{i},o_{i}\rangle$. The three elements in a triple denote subject, predicate and object, respectively. Given the knowledge base $\mathcal{F}$, our goal is to generate a natural language sentence $S$ which consists of a sequence of words $[w_{1},w_{2},\cdots,w_{M}]$, where $w_{m}$ denotes the $m$-th word in the sentence $S$. The generated sequence $S$ is required to be grammatically sound and correctly represent all the information contained in the knowledge base $\mathcal{F}$.

2.2. Sequence to Sequence Framework

Our work is based on the sequence to sequence framework (SEQ2SEQ). The standard sequence to sequence framework consists of an encoder and a decoder. Both of them are parameterized by recurrent neural networks (RNN).

The encoder takes in a sequence of discrete tokens $X=[x_{1},x_{2},\cdots,x_{L}]$. At $t$-th step, the encoder takes in a token and updates the hidden state recurrently:

[TABLE]

where $\mathbf{e}_{x_{t}}$ denotes the word embedding (Mikolov et al., 2013) of the $t$-th token. In general, $\mathbf{e}_{x_{t}}=W_{e}x_{t}$, where $W_{e}$ is a pre-trained or learned word embedding matrix with each column representing a embedding vector of a token; given $x_{t}$ is a one-hot vector, $W_{e}x_{t}$ get the corresponding column of $W_{e}$ for token $x_{t}$. $f^{\text{enc}}$ is a nonlinear function. Long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997) and gated recurrent unit (GRU) (Cho et al., 2014) are often considered as the paradigm of the function. The final output of encoder is an array of hidden states $\mathbf{H}^{\text{enc}}=[\mathbf{h}^{\text{enc}}_{1},\mathbf{h}^{\text{enc}}_{2},\cdots,\mathbf{h}^{\text{enc}}_{L}]$. Each hidden state can be regarded as a vector representation of all the previous tokens.

The decoder takes in the hidden states $\mathbf{H}^{\text{enc}}$ of the encoder as input and outputs a sequence of hidden states $\mathbf{H}^{\text{dec}}$. The hidden state at its $t$-th step is computed by:

[TABLE]

where $\mathbf{e}_{y_{t-1}}$ is the word embedding of the last output token of the decoder, and $\mathbf{e}_{y_{0}}$ is set to be a zero vector. $\mathbf{c}_{t}$ is a function of hidden states of encoder that provides the summary of the input sequence at step $t$, and typical choices include: i) $\mathbf{c}_{t}=\mathbf{h}^{\text{enc}}_{L}$; ii) the attention mechanism (Bahdanau et al., 2014) with $\mathbf{c}_{t}=g(\mathbf{h}^{\text{enc}}_{1},\mathbf{h}^{\text{enc}}_{2},\cdots,\mathbf{h}^{\text{enc}}_{L},\mathbf{h}_{t-1}^{\text{dec}})$.

In general, generation of language is modeled as an autoregressive sequential generation process with each token sampled from the probability distribution conditioning on its previous tokens. The probability distribution of $t$-th token is parameterized by a softmax over an affine transformation of the decoder’s hidden state at step $t$, i.e.

[TABLE]

where $W$ and $b$ are the weight matrix and the bias vector of output layer respectively, and $y_{<t}$ denotes the first $t-1$ tokens of target content. The probability distribution of the entire output sequence $Y$ conditioning on the input sequence $X$ is thus modeled as

[TABLE]

2.3. Maximum Likelihood Estimation (MLE)

Training neural language models through maximum likelihood estimation (MLE) is the most widely used method. The objective is equivalent to minimizing the cross entropy between the real data distribution $P$ and the estimated probability distribution $G_{\theta}$ by the generative models:

[TABLE]

where $Y$ denotes a complete sequence sampled from the real data distribution $P$ and $H$ denotes the cross entropy.

Maximizing Eq. 5 is equivalent to minimizing the Kullback-Leibler (KL) divergence between target distribution $P$ and learned distribution $G_{\theta}$, which is defined as

[TABLE]

where $H(P)$ is a constant irrelevant to parameter $\theta$.

For clarity, we here ignore the conditional context $X$ here. We will later regard the maximum likelihood estimation as minimizing the Kullback-Leibler (KL) divergence.

3. Objective Analysis

In this section, we will give a detailed discussion on the fundamental problems of minimizing KL divergence in training and explain why we choose the inverse KL divergence as our optimization objective. We will also discuss several related solutions.

3.1. Practical Tendency of KL and Inverse KL

The KL divergence between two distributions $P$ and $G_{\theta}$ is formulated as

[TABLE]

Since the KL divergence is non-negative, it is minimized when $G_{\theta}=P$. Unfortunately, in real-world scenarios, the target $P$ is usually a very complex distribution. Given limited capacity, the learned probabilistic model $G_{\theta}$ may only be a rough approximation.

As pointed out in (Arjovsky and Bottou, 2017), $\mathrm{KL}(P\|G_{\theta})$ goes to infinity if $P(Y)>0$ and $G_{\theta}(Y)\rightarrow 0$, which means that the cost function is extremely high when the distribution of generator fails to cover some patterns of the real data. On the other hand, the cost function is relatively low when the generator is low-quality samples, as $\mathrm{KL}(P\|G_{\theta})$ goes to zero if $G_{\theta}(Y)>0$ and $P(Y)\rightarrow 0$.

That is, although the optimal is guaranteed to be $G_{\theta}=P$ under MLE objective, during training, the estimated distribution $G_{\theta}(y)$ is more likely to have a wide coverage and possibly contain samples out of the real distribution, as illustrated in Figure 2(b). In practice, models trained via MLE have a high probability of generating rarely-seen sequences, most of which are inconsistent with human expressions due to exposure bias (Bengio et al., 2015).

With a similar argument to the behavioral tendency of $\mathrm{KL}(P\|G_{\theta})$, it can be shown that $\mathrm{KL}(G_{\theta}\|P)$ has less penalty to ”mode collapse”, which means $G$ tend to generate a family of similar samples. By contrast, $\mathrm{KL}(G_{\theta}\|P)$ assigns a large penalty to fake-looking samples. The typical non-optimal estimation, as illustrated in Figure 2(c), is that it covers several major modes of the real distribution, but misses several minor modes.

We here argue that in the conditional language generation task, especially such the triple-to-text tasks, minimizing the inverse KL divergence would be more preferred. Because, in these translation tasks, people usually care more about the quality of the generated text, rather than their diversity. In other words, it is tolerable to have low diversity, but it is usually unacceptable to be grammatically incorrect or miss important information.

3.2. The Decomposed Objective of Inverse KL

Here, we explain the property of inverse KL divergence via objective decomposition. We will show that minimizing the inverse KL divergence $\mathrm{KL}(G_{\theta}\|P)$ can be regarded as a direct optimization of the performance of the Turing test.

In Turing test , we assume that the human judges know the accurate natural language distribution $P$.(Mahoney, 1999) Given a language sample $X$, its quality is scored by $P(X)$. Thus the averaged score in a Turing test can be modeled as the negative cross-entropy $-H(G_{\theta},P)=\mathbb{E}_{Y\sim G_{\theta}}[\log P(Y)]$.

The inverse KL divergence can be rewritten as

[TABLE]

Eq. 8 illustrates that the objective of minimizing an inverse KL divergence can be be decomposed into two parts:

• •

Minimizing $H(G_{\theta},P)$, which corresponds to the objective of Turing test.

• •

Maximizing $H(G_{\theta})$, the self-entropy of the generator. It helps expand the support of $G_{\theta}$, to avoid disjoint support between $P$ and $G_{\theta}$, which may lead to gradient vanish problem (Arjovsky and Bottou, 2017).

3.3. Estimation of the Real Distribution $P$

In most real applications, $P$ is an empirical distribution and not directly accessible. For this reason we could not directly optimize the inverse KL divergence. In our proposed method, we introduce a new module $M_{\phi}$, called judger, to approximate target distribution $P$. The judger is trained via maximum likelihood estimation and the objective function for $M_{\phi}$ is

[TABLE]

Note that, although the judger distribution $M_{\phi}$ might suffer from the problems as mentioned earlier of MLE, i.e., it does not precisely model all the modes, it generally widely covers the distribution with the major modes having large probability masses. Then, based on this inaccurate estimated distribution $M_{\phi}$, we minimize the inverse KL divergence $\mathrm{KL}(G_{\theta}\|M_{\phi})$. As we discussed before, the inverse KL divergence cares more about the major modes and tends to ignore these minor modes, including small fake modes stemming from imperfect MLE estimation, so the shortcoming of MLE-based estimated distribution $M_{\phi}$ poses no serious problems here.

It is also important to notice that, if the two steps in our algorithm both get the optimum, we have $G_{\theta}=P$, which is the same as previous methods. The key benefit of our algorithm is that when it does not get the optimum, the generated samples still tend to be feasible.

3.4. JS Divergence: GANs and CoT

Some previous works also recognized the limitations of KL divergence and alleviated this problem with various optimization methods. Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) introduced a module named discriminator to distinguish whether a sample is from the real distribution or is forged by $G_{\theta}$. In theory, given a perfect discriminator, the training objective of GAN is equivalent to minimize Jensen-Shannon Divergence, which is defined as the symmetrized version of two aforementioned divergences:

[TABLE]

where $M=\frac{1}{2}(P+G_{\theta})$ is the average of two distributions.

GAN is initially designed for generating continuous data, which is not directly applicable to discrete tokens, such as language sentences. SeqGAN (Yu et al., 2017) is introduced to generate discrete tokens via adversarial training. However, the generative models trained by SeqGAN tend to have high variance due to the REINFORCE algorithm (Williams, 1992).

Another attempt to leverage Jensen-Shannon divergence on sequence generation tasks is CoT (Lu et al., 2018). CoT introduces a new module, the mediator $M_{\phi}$, which estimates the mixture data distribution $\frac{1}{2}(P+G_{\theta})$ via maximum likelihood estimation. Then $M_{\phi}$ is used to guide the training of the generator $G_{\theta}$ with JSD. However, in practice, we find that the optimization of $M_{\phi}$ could be problematic. According to our experiments, as the real distribution becomes complicated, $M_{\phi}$ tends to get a distribution to fit $G_{\theta}$ rather than accurately modeling the $\frac{1}{2}(P+G_{\theta})$. We explain this phenomenon as follows.

The real distribution $P$ is relatively complex, and the estimated distribution $G_{\theta}$ tends to be simple and smooth. Because of the wide coverage tendency of MLE, $M_{\phi}$ would cover $\frac{1}{2}(P+G_{\theta})$ in general; while due to limited capacity of $M_{\phi}$, $M_{\phi}$ tends to fit the simple one, i.e., $G_{\theta}$.

The problem that $M_{\phi}$ captures limited differences between $P$ and $G_{\theta}$ makes the training hard to converge.

Note that one key difference is that the target mediator distribution in CoT is dynamical and involves with the learning distribution $G_{\theta}$, while the judger in our method is estimating the static distribution $P$.

4. Methodology

In this section, we will first explain how to convert the task into a sequence-to-sequence generation problem, and then illustrate the details of how to optimize it with inverse KL divergence.

4.1. General Framework

The SEQ2SEQ framework cannot process graph-based data like RDF triples directly. Thus we first use a pre-processing technique similar to the one mentioned in (Trisedya et al., 2018). It substitutes the subjects and objects in the RDF triples and their corresponding entities in the sentences into their types.

For example, given a knowledge base [(”Bill Gates”, ”founder”, ”Microsoft Corporation”) , (”Microsoft Corporation”, ”startDate”, ”April 4, 1975” )] and its corresponding human-annotated natural sentence ”Bill Gates founded the Microsoft Corporation in April 4, 1975”. The pre-process module will map ”Bill Gates,” ”Microsoft Corporation” and ”April 4, 1975” into ”PERSON,” ”CORPORATION” and ”DATE” respectively, in both RDFs and the corresponding sentences. The pre-process can reduce the size of the vocabulary list, and improve the generalization capacity of the model so that it can handle most dates rather than just ”April 4, 1975”. Considering that the nodes in the knowledge bases are unordered, we also apply permutation among the triples to enhance the training data, and we believe this approach can improve the generalization capabilities of the final generation model.

The pre-processed RDF triples are then transformed into a sequence of discrete tokens. We use commas to separate elements within an RDF triple, and semicolons to separate different RDFs. For instance, the knowledge base mentioned above is turned into ”PERSON, founder, CORPORATION; CORPORATION, startDate, DATE”. Simultaneously, zero padding is used to fill all sequences into the same length.

Finally, a SEQ2SEQ method introduced in Section 2.2 is used to encode the processed triple and then translate it into a human-understandable sentence. To enhance the performance of the encoder-decoder model, attention mechanisms (Bahdanau et al., 2014) are used in our proposed framework. Figure 3 illustrates the general structure of this method.

4.2. Algorithm Details

The general idea of the proposed method is that: a module $M_{\phi}$ called judger is introduced to approximate the target distribution $P$, which is trained via maximum likelihood estimation. Based on the approximated distribution $M_{\phi}$, we then minimize the inverse KL divergence $\mathrm{KL}(G_{\theta}\|M_{\phi})$. The overall process is illustrated in Figure 4.

Because we target at the sequence to sequence translation task, the distribution of generator is modeled as a chain product of probability distribution of the next token $y_{t}$ conditioning on the input sequence $X$ and prefix $y_{<t}$,

[TABLE]

Within our framework, the judger is trained to model the target distribution via maximum likelihood estimation. The judger $M_{\phi}$ is also modeled as a chain product of conditional distributions,

[TABLE]

The objective function for $M_{\phi}$ is

[TABLE]

Given $M_{\phi}$ which estimates the real distribution $P$, we then update $G_{\theta}$ via minimizing the inverse KL divergence $\mathrm{KL}(G_{\theta}\,\|M_{\phi})$:

[TABLE]

where $(X,Y)\sim G_{\theta}$ denotes the data pair where $X$ is sampled from conditional context and $Y$ is the output of generator given $X$ as input. The objective can be directly optimized by taking Eq. (11) and  (12) into Eq (14), which can be reformulated as

[TABLE]

Algorithm 1 illustrated the overall algorithm of our proposed method. Note that instead of training the judger to convergence at the beginning, the judger and the generator are trained alternately. From the perspective of curriculum learning (Bengio et al., 2009), by gradually increasing the complexity of the generator’s training objective, it improves the generalization ability of the generator and helps find a better local optimum. Our method shares the same computational complexity as MLE training.

5. Experiments

5.1. Datasets

Our methods are evaluated on the following datasets.

WebNLG (Gardent et al., 2017) is extracted from 15 different DBPedia (Auer et al., 2007) categories, which consists of 25,298 (data, text) pairs and 9,674 distinct data units. The data units are sets of RDF triples, and the texts are sequences of one or more sentences verbalizing these data units. It also provides a set of 373 distinct RDF properties.

SemEval-2010 Task 8 (Hendrickx et al., 2009) was originally designed for multi-way classification of semantic relations between pairs of nominals. It contains 10,717 samples, divided as 8,000 for training and 2,717 for testing. The dataset contains nine relation types. Since each example is a sentence annotated for a pair of entities and the corresponding relation class for this entity pair in this dataset, we can extract an RDF triple from each sentence.

Baidu SKE222http://ai.baidu.com/broad/introduction is a large-scale human annotated dataset with more than 410,000 triples in over 200,000 real-world Chinese sentences, bounded by a pre-specified schema with 50 types of predicates. Each sample in SKE contains one sentence and a set of associated tuples. SKE Tuples are expressed in forms of (subject, predicate, object, subject type, object type). In our experiments, we only use knowledge bases related to Film and TV works domain, and each Chinese character is treated as a distinct token.

We select some data in the three data sets and divide them into a training set and a test set. Table 2 shows some statistical details about the data.

5.2. Implementation Details

The generator consists of a word embedding matrix, an encoder, a decoder, and the output layer. For the word embedding, we maintain two different sets of embeddings for encoder and decoder respectively; both are of 64 dimensions. Both encoder and decoder are built as an LSTM (Hochreiter and Schmidhuber, 1997) with hidden units of 128 dimensions. The dimension of hidden units of the output layer is also 128. We apply Bahdanau attention (Bahdanau et al., 2014) to the context vector $c_{t}$, which is computed as the weighted sum of encoder states. For the judger, we use the same configuration as the generator.

For the initialization, all initial parameters follow a standard Gaussian distribution $\mathcal{N}(0,1)$. All models are optimized using Adam optimization (Kingma and Ba, 2014) with a learning rate of 0.001 and a batch size of 64. The hyper-parameters of $g$ and $m$ in the algorithm are both set as 1, which makes the objective of the generator gradually harder as indicated in Section 4.2. We also pre-train the generator via MLE with the number pre-train epochs set as 2.

5.3. Baseline Algorithms

We validate our proposed method for RDF triple-to-text (we will later refer to as T2T) by comparing it with the following baselines. To give a fair comparison, we apply the same RDF pre-processing technique discussed in Section 4.1 to all the baselines.

• •

MLE. A common method for training sequence to sequence framework. For a fair comparison, the parameter setting of the generator is the same with our model.

• •

CoT. We adapt CoT (Lu et al., 2018) into conditional sequence generation task. As its authors suggested, the size of the hidden unit of the mediator is twice the size of the generator.

• •

Pointer-Generator Network (PG). See et al. (See et al., 2017) proposed pointer-generator network. Their work can be regarded as a combination of SEQ2SEQ and pointer network (Vinyals et al., 2015).

• •

SeqGAN. Yu et al. (Yu et al., 2017) used an adversarial network to provide the reward and train a sequence generator with policy gradient. According to (Li et al., 2017) and our initial experiments, in SEQ2SEQ framework, when the discriminator is parameterized as a convolutional neural network, it is difficult for the discriminator in SeqGAN to improve the generator. We thus follow (Li et al., 2017) and adapt the discriminator into a hierarchical recurrent neural network (Li et al., 2015).

• •

Neural Wikipedian (NW). Vougiouklis et al. (Vougiouklis et al., 2018) used a standard feed-forward neural network to encode RDF triples. Then the vectors derived from encoders are concatenated and used as the input of the decoder which generates summaries for RDF triples.

5.4. Metrics

For natural language generation tasks, the most widely accepted metric is human evaluation (Belz and Reiter, 2006). While human evaluation is reliable, it is hardly applied to quality evaluating of large corpus since it will involve too many human resources. Therefore, we have to introduce automatic metrics for evaluating all the sentences our system has generated. However, to our knowledge, no single automatic evaluation metric is sufficient to measure the performance of a natural language generation system (Novikova et al., 2017). Thus, in order to give objective results, we use a variety of automatic metrics to compare our models and benchmarks.

We have adopted three widely used word-level metrics: BLEU (Papineni et al., 2002), TER (Snover et al., 2006) and METEOR (Banerjee and Lavie, 2005). BLEU and METEOR333We use METEOR 1.5 (https://www.cs.cmu.edu/ alavie/METEOR/README.html), with parameters suggested by Denkowski et al. (Denkowski and Lavie, 2014) for universal evaluation to calculate the number of $n$-grams of the generated sentence occurs within the set of references.

Besides the traditional word-based metrics, we also evaluate the generator via likelihood and perplexity. Inspired by likelihood-based discrimination(McLachlan, 2004), we design a new metric which we refer to as ”predicate accuracy”. In detail, given a single RDF triple $X_{p}=\langle s,p,o\rangle$, a natural sentence $Y$ describing the triple and a generative model $G_{\theta}$, we can calculate $G_{\theta}(Y|X_{p})$, i.e. the predictive likelihood of the target sentence. If we keep subject $s$ and object $o$ unchanged, and substitute predicate $p$ with another predicate $p^{\prime}$, then our generative model can derive a probability density $G_{\theta}(Y|X_{p^{\prime}})$ for each predicate class $p^{\prime}$, where $X_{p^{\prime}}$ denotes the triple $\langle s,p^{\prime},o\rangle$. Then, we can use the likelihood of generative model to predict the predicate given subject $s$, object $o$ and sentence $X_{p}$, the predicted predicate $\hat{p}$ is

[TABLE]

where $P$ is the set of all kinds of predicate. The ”predicate accuracy” is defined as precision of $\hat{p}$ in Eq. 16 being the correct predicate describing the sentence $Y$.

We also use forward perplexity ($\mathtt{FPPL}$) to evaluate the quality of the generated text. Different from the traditional perplexity evaluated only on generative models, $\mathtt{FPPL}$ evaluate perplexity of generated samples from generator $G_{\theta}$ using another language model (denoted as $H_{\psi}$ ) trained on real data via MLE. According to Zhao et al. (Kim et al., 2017), $\mathtt{FPPL}$ measures the fluency of generated sentences.

[TABLE]

In our experiments, $H_{\psi}$ is implemented as an LSTM-based SEQ2SEQ model, whose word embedding size is set as 64, encoder hidden unit and decoder hidden unit is all set as 300.

5.5. Experiments Results

Table 3 shows the overall results of each training method on BLEU, TER, and METEOR among the three datasets. From the results, we find that our proposed T2T method improves the quality of the generated sentence on these word-based metrics. As we have analyzed, generators optimized via inverse KL divergence tend to generate text with more common expressions, while other baselines tend to use some low-quality text. Thus, sentences from T2T will overlap more words with reference text, which means it can achieve better performance on the word based metrics like BLEU.

The experiments on $\mathtt{FPPL}$ also validated our conclusion. Table 5(b) shows the results of $\mathtt{FPPL}$ on WebNLG dataset. Low forward perplexity validates that our method allows the generator to generate high-frequency language patterns more and better. We plot the training curves of $\mathtt{FPPL}$.

Table 4 shows the predicate accuracy of different training methods on SemEval datasets. Our model can fit the logical connection between real sentences and RDF predicates better compared with baselines.

Human evaluation is conducted on WebNLG dataset to validate the performance of our framework further. We choose WebNLG dataset because it consists of more RDF triples and its reference sentences are relatively simple. We randomly select 20 RDF triples from the dataset, along with the corresponding sentences generated by T2T and baselines. Ten human volunteers are asked to rate the sentences from two aspects: grammar and correctness. The score on grammar is used to judge whether the sentence contains grammatical errors, improper use of words and repetition. Correctness measures whether the sentence accurately represents the information in the RDF triples. The score for each criterion takes an integer between 1 and 10. Volunteers are given both scoring criteria and examples. Table 6 lists the results of overall human evaluation score.

Table 7 presents samples of generated sentences from different baselines and T2T given a knowledge base about Amatriciana sauce. Compared with baselines, the text generated from T2T is not only grammatically sound and correctly expresses all the information from RDF triples as well. We also found that when the length of the generated sentence is long, the quality of output from SeqGAN is compromised, which may because they use Monte Carlo sampling to guide the generator, which will introduce variance. The sentences generated by MLE correctly express the knowledge, but the grammar and the words are not quite authentic. Text generated using Pointer-Generator suffers from repetition. Neural Wikipedian can hardly express all information soundly given multiple triples.

6. Related Works

Our task can be regarded as a combination of two problems. One is on the training of neural language models; another is on converting knowledge bases (structured data) into natural languages.

6.1. Knowledge Base to Natural Language

Previous approaches on generating natural language from knowledge bases can be categorized into the following types: rule-based, template-based and neural language model based.

Generating sentences based on knowledge bases with hand-crafted rules is the main technology in traditional NLG systems, which often involves domain-specific knowledge and only works for a particular data type. Bontcheva et al. (Bontcheva and Wilks, 2004) designed a set of rules to generate natural language reports from medical data automatically. O’Donnell et al. (O’Donnell et al., 2000) designed a text generation system by utilizing the potential rules from relational databases. They specified the semantics of relational databases and reconstructed an ”Intelligent Labelling Explorer” (ILEX) system. Based on that, the ILEX system can interpret entities from databases based on information like domain taxonomy and specification of the data type. Cimiano et al. (Cimiano et al., 2013) presented a principled language generation architecture by analyzing statistical information derived from a domain corpus. Their system can write recipes based on RDF representations of a cooking domain. They mainly focus on extracting lexicon and then formulate the recipes with a parse tree.

Template-based generation is another traditional approach to convert structured data into text. In general, developing such kind of system often requires complex design about grammar, semantic and lexicalization (Deemter et al., 2005). Kukich (Kukich, 1983) designed a knowledge-based report generator which infers semantic messages from the data and then maps that information into a grammar-based template. Flanigan et al. (Flanigan et al., 2016) proposed a two-stage method for natural language generation from Abstract Meaning Representation (Banarescu et al., 2013). Duma et al. (Duma and Klein, 2013) formulated a system which automatically learns sentence templates using the corpus extracted from Simple English Wikipedia and DBpedia.

The former two technologies have good availability, reliability and do not rely on large quantities of corpora to train the model. However, they require a labor expert and have poor scalability.

6.2. Neural Language Models

Sequence-to-sequence model (Dušek, 2016) adopts an end-to-end generation method that converts a meaning representation into a sentence. As attention mechanism (Bahdanau et al., 2014) presents advantages in soft-searching the most relevant information among a sequence in neural machine translation task, Nallapati et al. (Nallapati et al., 2016) proposed a sequence-to-sequence attentional model to tackle text summarization task. See at al. (See et al., 2017) proposes a hybrid pointer-generator network facilitating copying words from the source text via pointing (Vinyals et al., 2015) while retaining the ability to produce new words via generator, and uses coverage to discourage repetition. Yu et al. (Yu et al., 2017) proposed SeqGAN framework that introduces GAN discriminator (Goodfellow et al., 2014) to provide the reward signal and uses policy gradient technique (Sutton et al., 2000) to bypass the generator differentiation problem. Lu et al. (Lu et al., 2018) proposed Cooperative Training (CoT) that coordinately trains a generative module and an auxiliary predictive module, to optimize the estimated Jensen-Shannon divergence.

Besides the studies on design and training of language models, the researchers also proposed many indicators for evaluating the quality of samples generated by language models. These metrics can be classified into word-based metrics and grammar-based metrics. Word-based metrics move from simple $n$-gram overlap (including BLEU, TER (Snover et al., 2006), ROUGE (Lin, 2004), NIST (Doddington, 2002), LEPOR (Han et al., 2012), CIDER (Vedantam et al., 2015) and METEOR (Banerjee and Lavie, 2005)) to semantic similarity like Semantic Text Similarity (Han et al., 2013). Grammar-based metrics include F-score, MaxMatch (Dahlmeier and Ng, 2012), I-measure (Felice and Briscoe, 2015). Besides, instead of comparing sentences words by words, EmbSim (Zhu et al., 2018) compares the word embeddings. Some metrics are likelihood-based metrics that estimate the cross-entropy between the generated sentences and the true data, such as $\text{NLL}_{\text{oracle}}$ (Yu et al., 2017) that estimates average negative log-likelihood of generated sentences on oracle LSTM.

7. Conclusion

In this paper, we studied the problem of converting knowledge base RDF triples into natural languages. To handle this problem, we formulated it as a conditional natural language problem and utilized the discrete sequence generative models. We analyzed the limitations of existing methods on conditional sequence generative models and proposed a new method T2T which approximately optimizes an inverse Kullback-Leibler divergence between the real distribution and the learned one. We validated the proposed method on three benchmark datasets. The experiment results show that our method outperforms the baselines.

Our model is not limited in the task of translating knowledge bases RDF triples to natural languages; it can also be applied to other conditional generation tasks like machine translation and question answering systems, which we leave as future work.

Acknowledgements.

The work is sponsored by Huawei Innovation Research Program. The corresponding author Weinan Zhang thanks the support of National Natural Science Foundation of China (61702327, 61772333, 61632017), Shanghai Sailing Program (17YF1428200). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor.