Better, Faster, Stronger Sequence Tagging Constituent Parsers

TL;DR

This paper introduces novel techniques to improve sequence tagging constituent parsers, making them more accurate and faster, and achieves state-of-the-art results across multiple languages and datasets.

Contribution

The paper presents a combination of training strategies, label decomposition, and auxiliary losses to enhance sequence tagging parsers for constituent parsing.

Findings

Surpassed previous sequence tagging parsers on English and Chinese Penn Treebanks.

Achieved state-of-the-art results on Basque, Hebrew, Polish, and Swedish datasets.

Reduced parsing time while improving accuracy.

Abstract

Sequence tagging models for constituent parsing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propagation arising from greedy decoding. To effectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gradient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further. On the SPMRL datasets, we observe even greater improvements across the board, including a new state of the art on Basque, Hebrew, Polish and Swedish.

Tables

Table 1. Table 1: Results on the ptb dev set, compared against Gómez-Rodríguez and Vilares ( 2018 ) . de refers to dynamic encoding and mtl to a model that additionally casts the problem as multi-task learning. Each auxiliary task is added separately to the baseline with de and mtl . Policy gradient fine-tunes the model that includes the best auxiliary task.

Model	F-score	(+/-)	Sents/s
Gómez and Vilares (2018)	90.60	-	109
Our baseline	90.64	(+0.04)	111
+ de	91.16	(+0.56)	111
+ mtl	91.27	(+0.67)	130
aux($n_{t+1}$)	90.19	(+0.59)	130
aux($n_{t-1}$)	91.40	(+0.80)	130
aux(distances)	91.44	(+0.84)	130
+ pg	91.67	(+1.07)	130

Table 2. Table 2: Results on the ctb and spmrl dev sets

Model	ctb	Basque	French	German	Hebrew	Hungarian	Korean	Polish	Swedish
Our baseline	88.57	87.62	80.19	86.48	89.09	88.61	82.79	92.60	78.82
+de	88.37	87.57	80.27	87.44	88.82	88.42	83.11	93.35	78.24
+mtl	88.57	89.12	80.84	87.54	92.63	89.55	83.27	93.81	81.71
aux($n_{t+1}$)	88.73	89.37	81.09	87.59	92.57	89.50	83.28	93.86	81.70
aux($n_{t-1}$)	88.48	89.19	80.91	87.67	92.45	89.52	83.37	93.87	81.61
aux(distances)	88.51	89.23	81.17	87.68	92.56	89.58	83.39	93.83	82.02
+pg	89.01	89.44	81.28	87.83	92.56	89.63	83.63	93.93	82.05

Table 3. Table 3: Comparison on the ptb test set. Kitaev and Klein ( 2018b ) are results published after this work was submitted (italics represent the cases where they obtain a new state of the art on the corresponding language).

Model	Sents/s	Hardware	F-score
Vinyals et al. (2015)	120	Many CPU	88.30
Coavoux and Crabbé (2016)	168	1 CPU	88.60
Fernández and Martins (2018)	41	1 CPU	90.20
Zhu et al. (2013)	90	1 CPU	90.40
Dyer et al. (2016)	17	1 CPU	91.20
Stern et al. (2017)	76	16 CPU	91.77
Shen et al. (2018)	111	1 GPU	91.80
Kitaev and Klein (2018a)	213	2 GPU	93.55
(single model)
Kitaev and Klein (2018a)	71	2 GPU	95.13
(with elmo)
Kitaev and Klein (2018b)	-	-	95.77
(ensemble and bert)
Gómez and Vilares (2018)	115	1 CPU	90.70
Our baseline	115	1 CPU	90.75
+de	115	1 CPU	90.85
+mtl	132	1 CPU	90.97
+ best aux	132	1 CPU	90.97
+pg	132	1 CPU	91.13
+pg	942	1 GPU	91.13
+pg (no char emb)	149	1 CPU	91.09
+pg (no char emb)	1267	1 GPU	91.09

Table 4. Table 4: Comparison on the ctb test set

Model	F-score
Zhu et al. (2013)	83.2
Dyer et al. (2016)	84.6
Liu and Zhang (2017)	86.1
Shen et al. (2018)	86.5
Fernández and Gómez-Rodríguez (2018)	86.8
Gómez and Vilares (2018)	84.1
Our baseline	83.90
+de	83.98
+mtl	84.24
+best aux	85.01
+pg	85.61
+pg (no char emb)	83.93

Table 5. Table 5: Comparison on the test spmrl datasets (except Arabic). Kitaev and Klein ( 2018b ) are results published after this work was submitted (italics represent the cases where they obtain a new state of the art on the corresponding language).

Model	Basque	French	German	Hebrew	Hungarian	Korean	Polish	Swedish	Avg
Fernández-González and Martins (2015)	85.90	78.75	78.66	88.97	88.16	79.28	91.20	82.80	84.21
Coavoux and Crabbé (2016)	86.24	79.91	80.15	88.69	90.51	85.10	92.96	81.74	85.67
Björkelund et al. (2014) (ensemble)	88.24	82.53	81.66	89.80	91.72	83.81	90.50	85.50	86.72
Coavoux and Crabbé (2017)	88.81	82.49	85.34	89.87	92.34	86.04	93.64	84.00	87.82
Kitaev and Klein (2018a)	89.71	84.06	87.69	90.35	92.69	86.59	93.69	84.35	88.64
Kitaev and Klein (2018b) (with bert)	91.63	87.42	90.20	92.99	94.90	88.80	96.36	88.86	91.40
Baseline	89.20	79.58	82.33	88.67	90.10	82.63	92.48	82.40	85.92
+de	89.19	79.72	82.91	88.60	89.65	82.86	93.20	82.11	86.03
+mtl	90.60	80.02	83.48	91.91	90.32	83.11	93.80	85.19	87.30
+best aux	90.91	80.33	83.49	92.05	90.33	82.97	93.84	85.58	87.44
+pg	90.85	80.40	83.42	92.05	90.38	83.24	93.93	85.54	87.48
+pg (no char emb)	89.81	80.41	83.60	91.75	90.01	82.65	93.87	85.46	87.20

Table 6. Table 6: Comparison of speeds on the spmrl datasets

Dataset	Baseline	Full	Full (no char)
	speed	speed(increase)	speed(increase)
Basque	179	223 (1.25x)	257 (1.44x)
French	76	91 (1.20x)	104 (1.37x)
German	70	100 (1.43x)	108 (1.54x)
Hebrew	44	102 (2.32x)	115 (2.61x)
Hungarian	93	134 (1.44x)	150 (1.61x)
Korean	197	213 (1.08x)	230 (1.17x)
Polish	187	253 (1.35x)	278 (1.49x)
Swedish	98	158 (1.61x)	187 (1.81x)

Table 7. Table 7: Additional hyperparameters of the base model and Policy Gradient fine-tuning

Hyperparameter	Value
bilstm size	800
# bilstm layers	2
optimizer	sgd
loss	cat. cross-entropy
learning rate	0.2
decay (linear)	0.05
momentum	0.9
dropout	0.5
word emb size	100
features size	20
character emb size	30
batch size training	8
training epochs	100
batch size test	128
PG finetuning Hyperparameter	Value
# samples	8
learning rate	0.0005
entropy regularization coefficient	0.01
variance reduction burn-in # of examples	1000
layers frozen	word & char embeddings
noise process initial stddev	0.1
noise process desired action stddev	0.5
noise process adaptation coefficient	1.05

Equations

$${\Delta }_{\theta }log\pi (l_t|s_t;\theta )R_{tree}$$

$$\begin{gathered} {\Delta }_{\theta }(log\pi (l_t|s_t;\theta )+N)(R_{tree}-B_{tree}) \\ +\beta {\Delta }_{\theta }H(\pi (s_t;\theta )+N) \end{gathered}$$

Full text

This is a revised version of the paper originally published in NAACL 2019, with a corrigendum at the end describing the changes. The previous version contained a bug where the script evalb for comparison against the state-of-the-art was not considering the .prm parameter files.

1 Introduction

Constituent parsing is a core task in natural language processing (nlp), with a wide set of applications. Most competitive parsers are slow, however, to the extent that it is prohibitive of downstream applications in large-scale environments Kummerfeld et al. (2012). Previous efforts to obtain speed-ups have focused on creating more efficient versions of traditional shift-reduce Sagae and Lavie (2006); Zhang and Clark (2009) or chart-based parsers Collins (1997); Charniak (2000). Zhu et al. (2013), for example, presented a fast shift-reduce parser with transitions learned by a svm classifier. Similarly, Hall et al. (2014) introduced a fast gpu implementation for Petrov and Klein (2007), and Shen et al. (2018) significantly improved the speed of the Stern et al. (2017) greedy top-down algorithm, by learning to predict a list of syntactic distances that determine the order in which the sentence should be split.

In an alternative line of work, some authors have proposed new parsing paradigms that aim to both reduce the complexity of existing parsers and improve their speed. Vinyals et al. (2015) proposed a machine translation-inspired sequence-to-sequence approach to constituent parsing, where the input is the raw sentence, and the ‘translation’ is a parenthesized version of its tree. Gómez-Rodríguez and Vilares (2018) reduced constituent parsing to sequence tagging, where only $n$ tagging actions need to be made, and obtained one of the fastest parsers to date. However, the performance is well below the state of the art Dyer et al. (2016); Stern et al. (2017); Kitaev and Klein (2018a).

Contribution

We first explore different factors that prevent sequence tagging constituent parsers from obtaining better results. These include: high error rates when long constituents need to be closed, label sparsity, and error propagation arising from greedy inference. We then present the technical contributions of the work. To effectively close brackets of long constituents, we combine the relative-scale tagging scheme used by Gómez-Rodríguez and Vilares (2018) with a secondary top-down absolute-scale scheme. This makes it possible to train a model that learns how to switch between two encodings, depending on which one is more suitable at each time step. To reduce label sparsity, we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning (mtl) Caruana (1997), to decompose a large label space and also obtain speed ups. Finally, we mitigate error propagation using two strategies that come at no cost to inference efficiency: auxiliary tasks and policy gradient fine-tuning.

2 Preliminaries

We briefly introduce preliminaries that we will build upon in the rest of this paper: encoding functions for constituent trees, sequence tagging, multi-task learning, and reinforcement learning.

Notation

We use $w$=$[w_{0},w_{1},...,w_{n}]$ to refer to a raw input sentence and bold style lower-cased and math style upper-cased characters to refer to vectors and matrices, respectively (e.g. $\mathbf{x}$ and $\mathbf{W}$).

2.1 Constituent Parsing as Sequence Tagging

Gómez-Rodríguez and Vilares (2018) define a linearization function of the form $\Phi_{|w|}:T_{|w|}\rightarrow L^{(|w|-1)}$ to map a phrase structure tree with $|w|$ words to a sequence of labels of length $|w|-1$.222They (1) generate a dummy label for the last word and (2) pad sentences with a beginning- and end-of-sentence tokens. For each word $w_{t}$, the function generates a label $l_{t}\in L$ of the form $l_{t}$=$(n_{t},c_{t},u_{t})$, where:

• •

$n_{t}$ encodes the number of ancestors in common between between $w_{t}$ and $w_{t+1}$. To reduce the number of possible values, $n_{t}$ is encoded as the relative variation in the number of common ancestors with respect to $n_{t-1}$.

• •

$c_{t}$ encodes the lowest common ancestor between $w_{t}$ and $w_{t+1}$.

• •

$u_{t}$ contains the unary branch for $w_{t}$, if any.

Figure 1 explains the encoding with an example.

2.2 Sequence Tagging

Sequence tagging is a structured prediction task that generates an output label for every input token. Long short-term memory networks (lstm) Hochreiter and Schmidhuber (1997) are a popular architecture for such tasks, often giving state-of-the-art performance Reimers and Gurevych (2017); Yang and Zhang (2018).

Tagging with lstms

In lstms, the prediction for the $i$th element is conditioned on the output of the previous steps. Let lstmθ($\mathbf{x}_{1:n}$) be a parametrized function of the network, where the input is a sequence of vectors $\mathbf{x}_{1:n}$, its output is a sequence of hidden vectors $\mathbf{h}_{1:n}$. To obtain better contextualized hidden vectors, it is possible to instead use bidirectional lstms Schuster and Paliwal (1997). First, a lstm${}_{\theta}^{l}$ processes the tokens from left-to-right and then an independent lstm${}_{\theta}^{r}$ processes them from right-to-left. The $i$th final hidden vector is represented as the concatenation of both outputs, i.e. bilstm${}_{\theta}(\mathbf{x},i)$ = \textsc{lstm{}{\theta}^{l}}(\mathbf{x}_{[1:i]})\circ\textsc{lstm{}{\theta}^{r}}(\mathbf{x}_{[|\mathbf{x}|:i]}). bilstms can be stacked in order to obtain richer representations. To decode the final hidden vectors into discrete labels, a standard approach is to use a feed-forward network together with a softmax transformation, i.e. $P(y|\mathbf{h}_{i})$ = $softmax(W\cdot\mathbf{h}_{i}+\mathbf{b})$. We will use the bilstm-based model by Yang and Zhang (2018), for direct comparison against Gómez-Rodríguez and Vilares (2018), who use the same model. As input, we will use word embeddings, PoS-tag embeddings and a second word embedding learned by a character-based lstm layer. The model is optimized minimizing the categorical cross-entropy loss, i.e. $\mathcal{L}$ = $-\sum{log(P(y|\mathbf{h}_{i}))}$. The architecture is shown in Figure 2.

2.3 Multi-task Learning

Multi-task learning is used to solve multiple tasks using a single model architecture, with task-specific classifier functions from the outer-most representations Caruana (1997); Collobert and Weston (2008). The benefits are intuitive: sharing a common representation for different tasks acts as a generalization mechanism and allows to address them in a parallel fashion. The hard-sharing strategy is the most basic mtl architecture, where the internal representation is fully shared across all tasks. The approach has proven robust for a number of nlp tasks Bingel and Søgaard (2017) and comes with certain guarantees if a common, optimal representation exists Baxter (2000). Dong et al. (2015) use it for their multilingual machine translation system, where the encoder is a shared gated recurrent neural network Cho et al. (2014) and the decoder is language-specific. Plank et al. (2016) also use a hard-sharing setup to improve the performance of bilstm-based PoS taggers. To do so, they rely on auxiliary tasks, i.e, tasks that are not of interest themselves, but that are co-learned in a mtl setup with the goal of improving the network’s performance on the main task(s). We will introduce auxiliary tasks for sequence tagging constituent parsing later on in this work. A mtl architecture can also rely on partial sharing when the different tasks do not fully share the internal representations Duong et al. (2015); Rei (2017); Ruder et al. (2019) and recent work has also shown that hierarchical sharing (e.g. low-level task outputs used as input for higher-level ones) could be beneficial Søgaard and Goldberg (2016); Sanh et al. (2018).

2.4 Policy Gradient Fine-tuning

Policy gradient (pg) methods are a class of reinforcement learning algorithms that directly learn a parametrized policy, by which an agent selects actions based on the gradient of a scalar performance measure with respect to the policy. Compared to other reinforcement learning methods, pg is well-suited to nlp problems due to its appealing convergence properties and effectiveness in high-dimensional spaces Sutton and Barto (2018).

Previous work on constituent parsing has employed pg methods to mitigate the effect of exposure bias, finding that they function as a model-agnostic substitute for dynamic oracles (Fried and Klein, 2018). Similarly, Le and Fokkens (2017) apply pg methods to Chen and Manning (2014)’s transition-based dependency parser to reduce error propagation. In this work, we also employ pg to fine-tune models trained using supervised learning. However, our setting (sequence tagging) has a considerably larger action space than a transition parser. To deal with that, we will adopt a number of variance reduction and regularization techniques to make reinforcement learning stable.

3 Methods

We describe the methods introduced in this work, motivated by current limitations of existing sequence tagging models, which are first reviewed. The source code can be found as a part of https://github.com/aghie/tree2labels.

3.1 Motivation and Analysis

For brevity, we limit this analysis to the English Penn Treebank (ptb) Marcus et al. (1993). We reproduced the best setup by Gómez-Rodríguez and Vilares (2018), which we are using as baseline, and run the model on the development set. We below show insights for the elements of the output tuple $(n_{t},c_{t},u_{t})$, where $n_{t}$ is the number of levels in common between $w_{t}$ and $w_{t+1}$, $c_{t}$ is the non-terminal symbol shared at that level, and $u_{t}$ is a leaf unary chain located at $w_{t}$.

High error rate on closing brackets

We first focus on predicting relative tree levels ($n_{t}$). See Figure 3 for F-scores over $n_{t}$ labels. The sparsity on negative $n_{t}$s is larger than for the positive ones, and we see that consequently, the performance is also significantly worse for negative $n_{t}$ values, and performance worsens with higher negative values. This indicates that the current model cannot effectively identify the end of long constituents. This is a known source of error for shift-reduce or chart-based parsers, but in the case of sequence tagging parsers, the problem seems particularly serious.

Sparsity

The label space is large and sparse: the output labels are simply the possible values in the tuple $(n_{t},c_{t},u_{t})$. An analysis over the ptb training set shows a total of 1423 labels, with 58% of them occurring 5 or less times. These infrequent cases might be difficult to predict, even if some of the elements of the tuple are common.

Greedy decoding

Greedy decoding is prone to issues such as error propagation. This is a known source of error in transition-based dependency parsing Qi and Manning (2017); in contrast with graph-based parsing, in which parsing is reduced to global optimization over edge-factored scores McDonald et al. (2005).

In the case of bilstm-based sequence tagging parsers, for a given word $w_{t}$, the output label as encoded by Gómez-Rodríguez and Vilares (2018) only reflects a relation between $w_{t}$ and $w_{t+1}$. We hypothesize that even if the hidden vector representations are globally contextualized over the whole sequence, the intrinsic locality of the output label also turns into error propagation and consequently causes a drop in the performance. These hypotheses will be tested in §4. In particular, we will evaluate the impact of the different methods intended to perform structured inference (§3.4).

3.2 Dynamic Encodings

Gómez-Rodríguez and Vilares (2018) encode the number of common ancestors $n_{t}$, from the output tuple $(n_{t},c_{t},u_{t})$, as the variation with respect to $n_{t-1}$. We propose instead to encode certain elements of a sentence using a secondary linearization function. The aim is to generate a model that can dynamically switch between different tagging schemes at each time step $t$ to select the one that represents the relation between $w_{t}$ and $w_{t+1}$ in the most effective way.

On the one hand, the relative-scale encoding is effective to predict the beginning and the end of short constituents, i.e. when a short constituent must be predicted ($|n_{t}|\leq 2$). On the other hand, with a relative encoding scheme, the F-score was low for words where the corresponding $n_{t}$ has a large negative value (as showed in Figure 3). This matches a case where a long constituent must be closed: $w_{t}$ is located at a deep level in the tree and will only (probably) share a few ancestors with $w_{t+1}$. These configurations are encoded in a more sparse way by a relative scheme, as the $n_{t}$ value shows a large variability and it depends on the depth of the tree in the current time step. We can obtain a compressed representation of these cases by using a top-down absolute scale instead, as any pair of words that share the same $m$ top levels will be equally encoded. The absolute scale becomes however sparse when predicting deep levels. Figure 4 illustrates the strengths and weaknesses of both encodings with an example, and how a dynamically encoded tree helps reduce variability on $n_{t}$ values.

In our particular implementation, we will be using the following setup:

• •

$\Phi_{|w|}:T_{|w|}\rightarrow L^{|w|-1}$, the relative-scale encoding function, is used by default.

• •

$\Omega_{|w|}:T_{|w|}\rightarrow L^{\prime|w|-1}$ is the secondary linearization function that maps words to labels according to a top-down absolute scale. $\Omega$ is used iff: (1) $\Omega(w_{[t:t+1]})$ = $(n^{\prime}_{t},c^{\prime}_{t},u^{\prime}_{t})$ with $n^{\prime}_{t}\leq 3$, i.e. $w_{t}$ and $w_{t+1}$ share at most the three top levels, and (2) $\Phi(w_{[t:t+1]})$ = $(n_{t},c_{t},u_{t})$ with $n_{t}\leq-2$, i.e. $w_{t}$ is at least located two levels deeper in the tree than $w_{t+1}$.333The values were selected based on the preliminary experiments of Figure 3.

3.3 Decomposition of the label space

We showed that labels of the form $(n_{t},c_{t},u_{t})\in L$ are sparse. An intuitive approach is to decompose the label space into three smaller sub-spaces, such that $n_{i}\in N$, $c_{i}\in C$ and $u_{i}\in U$. This reduces the output space from potentially $|N|\times|C|\times|U|$ labels to just $|N|+|C|+|U|$. We propose to learn this decomposed label space through a multi-task learning setup, where each of the subspaces is considered a different task, namely taskN, taskC and taskU. The final loss is now computed as $\mathcal{L}=\mathcal{L}_{n}+\mathcal{L}_{c}+\mathcal{L}_{u}$.

We relied on a hard-sharing architecture, as it has been proved to reduce the risk of overfitting the shared parameters Baxter (1997). A natural issue that arises is that the prediction of labels from different label sub-spaces could be interdependent to a certain extent, and therefore a hierarchical sharing architecture could also be appropriate. To test this, in preliminary experiments we considered variants of hierarchical sharing architectures. We fed the output of the taskU as input to taskN and/or taskC. Similarly, we tested whether it was beneficial to feed the output of taskN into taskC, and viceversa. However, all these results did not improve those of the hard-sharing model. In this context, in addition to a generalization mechanism, the shared representation could be also acting as way to keep the model aware of the potential interdependencies that might exist between subtasks.

3.4 Mitigating Effects of Greedy Decoding

We propose two ways to mitigate error propagation arising from greedy decoding in constituent parsing as sequence tagging: auxiliary tasks and policy gradient fine-tuning. Note that we want to optimize bracketing F-score and speed. For this reason we do not explore approaches that come at a speed cost in testing time, such as beam-search or using conditional random fields Lafferty et al. (2001) on top of our lstm.

Auxiliary tasks

Auxiliary tasks force the model to take into account patterns in the input space that can be useful to solve the main task(s), but that remain ignored due to a number of factors, such as the distribution of the output label space Rei (2017). In a similar fashion, we use auxiliary tasks as a way to force the parser to pay attention to aspects beyond those needed for greedy decoding. We propose and evaluate two separate strategies:

Predict partial labels $n_{t+k}$ that are $k$ steps from the current time step $t$. This way we can jointly optimize at each time step a prediction for the pairs $(w_{t},w_{t+1})$, …, $(w_{t+k},w_{t+k+1})$. In particular, we will experiment both with previous and upcoming $n_{k}$’s, setting $|k|$=$1$. 2. 2.

Predict the syntactic distances presented by Shen et al. (2018), which reflect the order a sentence must be split to obtain its constituent tree using a top-down parsing algorithm Stern et al. (2017). The algorithm was initially defined for binary trees, but its adaptation to n-ary trees is immediate: leaf nodes have a split priority of zero and the ancestors’ priority is computed as the maximum priority of their children plus one. In this work, we use this algorithm in a sequence tagging setup: the label assigned to each token corresponds to the syntactic distance of the lowest common ancestor with the next token. This is illustrated in Figure 5.

The proposed auxiliary tasks provide different types of contextual information. On the one hand, the encoding of the $n_{t}$s by Gómez-Rodríguez and Vilares (2018) only needs to know about $w_{t}$ and $w_{t+1}$ paths to generate the label for the time step $t$. On the other hand, to compute the syntactic distance of a given non-terminal symbol, we need to compute the syntactic distances of its subtree, providing a more global, but also sparser context. For training, the loss coming from the auxiliary task(s) is weighted by $\beta$=0.1, i.e, the final loss is computed as $\mathcal{L}=\mathcal{L}_{n}+\mathcal{L}_{c}+\mathcal{L}_{u}+\beta\sum_{a}{\mathcal{L}_{a}}$.

Policy gradient fine-tuning

Policy gradient training methods allow us to fine-tune our models with a tree-level objective, optimizing directly for bracketing F-score. We start off with a converged supervised model as our initial policy. The sequence labeling model can be seen as a functional approximation of the policy $\pi$ parametrized by $\theta$, which at timestep $t$ selects a label $l_{t}$=$(n_{t},c_{t},u_{t})$4443 different labels in the mtl setting. given the current state of the model’s parameters, $s_{t}$. The agent’s reward, $R_{tree}$, is then derived from the bracketing F-score. This can be seen as a variant of the reinforce algorithm (Williams, 1992) where the policy is updated by gradient ascent in the direction of:

[TABLE]

Baseline and Variance Reduction

We use as baseline a copy of a pre-trained model where the parameters are frozen. The reward used to scale the policy gradient can then be seen as an estimate of the advantage of an action $l_{t}$ in state $s_{t}$ over the baseline model. This is equivalent to $R_{tree}-B_{tree}$, where $R_{tree}$ is the bracketing F-score of a sequence sampled from the current policy and $B_{tree}$ is the the tree-level F-score of the sequence greedily predicted by the baseline. To further reduce the variance, we standardize the gradient estimate $\Delta_{\theta}$ using its running mean and standard deviation for all candidates seen in training so far. In initial experiments without these augmentations, we observed that fine-tuning with vanilla pg often led to a deterioration in performance. To encourage exploration away from the converged supervised model’s policy, we add the entropy of the policy to the objective function (Williams and Peng, 1991). Moreover, following Lillicrap et al. (2015), we optionally add noise sampled from a noise process $N$ to the policy. The gradient of our full fine-tuning objective function takes the following form:

[TABLE]

where $H$ is the entropy and $\beta$ controls the strength of the entropy regularization term.

4 Experiments

We now review the impact of the proposed techniques on a wide variety of settings.

Datasets

We use the English Penn Treebank (ptb) Marcus et al. (1993) and the Chinese Penn Treebank (ctb) Xue et al. (2005). For these, we use the same predicted PoS tags as Dyer et al. (2016). We also provide detailed results on the spmrl treebanks Seddah et al. (2014),555Except for Arabic, for which we do not have the license. a set of datasets for constituent parsing on morphologically rich languages. For these, we use the predicted PoS tags provided together with the corpora. To the best of our knowledge, we provide the first evaluation on the spmrl datasets for sequence tagging constituent parsers.

Metrics

We report bracketing F-scores, using the evalb and the eval-spmrl scripts parametrized with the collins.prm and spmrl.prm files, respectively. We measure the speed in terms of sentences per second.

Setup

We use ncrfpp Yang and Zhang (2018), for direct comparison against Gómez-Rodríguez and Vilares (2018). We adopt bracketing F-score instead of label accuracy for model selection and report this performance as our second baseline. After 100 epochs, we select the model that fared best on the development set. We use GloVe embeddings Pennington et al. (2014) for our English models and zzgiga embeddings Liu and Zhang (2017) for the Chinese models, for a more homogeneous comparison against other parsers Dyer et al. (2016); Liu and Zhang (2017); Fernández-González and Gómez-Rodríguez (2018). ELMo Peters et al. (2018) or BERT Devlin et al. (2018) could be used to improve the precision, but in this paper we focus on keeping a good speed-accuracy tradeoff. For spmrl, no pretrained embeddings are used, following Kitaev and Klein (2018a). As a side note, if we wanted to improve the performance on these languages we could rely on the CoNLL 2018 shared task pretrained word embeddings Zeman et al. (2018) or even the multilingual bert model666https://github.com/google-research/bert/blob/master/multilingual.md. Our models are run on a single CPU777Intel Core i7-7700 CPU 4.2 GHz (and optionally on a consumer-grade GPU for further comparison) using a batch size of 128 for testing. Additional hyperparameters can be found in Appendix A.

4.1 Results

Table 1 contrasts the performance of our models against the baseline on the ptb development set.

To show that the model which employs dynamic encoding is better (+0.56) than the baseline when it comes to closing brackets from long constituents, we compare their F-scores in Figure 6. When we recast the constituent-parsing-as-sequence-tagging problem as multi-task learning, we obtain both a higher bracketing F-score (+0.67) and speed (1.17x faster). Fusing strategies to mitigate issues from greedy decoding also leads to better models (up to +0.84 when adding an auxiliary task888We observed that adding more than one auxiliary task did not translate into a clear improvement. We therefore chose the auxiliary task that performed the best in the development set. and up to +1.07 if we also fine-tune with pg). Note that including auxiliary tasks and pg come at a time cost in training, but not in testing, which makes them suitable for fast parsing.

Table 2 replicates the experiments on the ctb and the spmrl dev sets. The dynamic encoding improves the performance of the baseline on large treebanks, e.g. German, French or Korean, but causes some drops in the smaller ones, e.g. Swedish or Hebrew. Overall, casting the problem as multitask learning and the strategies used to mitigate error propagation lead to improvements.

For the experiments on the test sets we select the models that summarize our contributions: the models with dynamic encoding and the multi-task setup, the models including the best auxiliary task, and the models fine-tuned with policy gradient.

Tables 3, 4 and 5 compare our parsers against the state of the art on the ptb, ctb and spmrl test sets. Gómez-Rodríguez and Vilares (2018) also run experiments without character embeddings, to improve speed without suffering from a big drop in performance. For further comparison, we also include them as additional results (shadowed). In a related line, Smith et al. (2018) show that for dependency parsing two out of three embeddings (word, postag and characters) can suffice.

4.2 Discussion

The results across the board show that the dynamic encoding has a positive effect on 7 out of 10 treebanks. Casting the constituent-parsing-as-sequence-labeling problem as mtl surpasses the baseline for all tested treebanks (and it leads to better parsing speeds too). Finally, by mitigating issues from greedy decoding we further improve the performance of all models that include dynamic encodings and multi-task learning.

On the ptb, our models are both faster and more accurate than existing sequence tagging or sequence-to-sequence models, which already were among the fastest parsers Gómez-Rodríguez and Vilares (2018); Vinyals et al. (2015). We also outperform other approaches that were not surpassed by the original sequence tagging models in terms of F-score Zhu et al. (2013); Fernández-González and Martins (2015). On the ctb our techniques also have a positive effect. The baseline parses 70 sents/s on the ctb, while the full model processes up to 120. The speed up is expected to be larger than the one obtained for the ptb because the size of the label set for the baseline is bigger, and it is reduced in a greater proportion when the constituent-parsing-as-sequence-labeling problem is cast as mtl.

On the spmrl corpora, we provide the first evaluation of sequence labeling constituent parsers, to verify if these perform well on morphologically rich languages. We then evaluated whether the proposed techniques can generalize on heterogeneous settings. The tendency observed for the original tagging models by Gómez-Rodríguez and Vilares (2018) is similar to the one for the ptb and ctb: they improve other fast parsers, e.g. Coavoux and Crabbé (2016), in 3 out of 8 treebanks and Fernández-González and Martins (2015) in 6 out of 8, but their performance is below more powerful models. When incorporating the techniques presented in this work, we outperform the original sequence tagging models on all datasets. We outperform the current best model for Basque, Hebrew and Polish Kitaev and Klein (2018a) and for Swedish Björkelund et al. (2014), which corresponds to the four smallest treebanks among the spmrl datasets. This indicates that even if sequence tagging models are conceptually simple and fast, they can be very suitable when little training data is available. This is also of special interest in terms of research for low-resource languages. Again, casting the problem as mtl reduces the parsing time for all tested treebanks, as reflected in Table 6. Finally, for treebanks such as French, designing methods to handle multi-word expressions could lead to better results, getting closer to other parsers Coavoux and Crabbé (2017).

5 Conclusion

We have explored faster and more precise sequence tagging models for constituent parsing. We proposed a multitask-learning architecture that employs dynamic encodings, auxiliary tasks, and policy gradient fine-tuning. We performed experiments on the English and Chinese Penn Treebanks, and also on the spmrl datasets. Our models improve current sequence tagging parsers on all treebanks, both in terms of performance and speed. We also report state-of-the-art results for the Basque, Hebrew, Polish, and Swedish datasets. The methods presented in this work are specifically designed for constituent parsing. However, it seems natural to apply some of these to other nlp tagging tasks, e.g. using multi-task learning to predict sub-level morphological information for morphologically-rich part-of-speech tagging.

Acknowlegments

DV has received support from the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), from the TELEPARES-UDC project (FFI2014-51978-C2-2-R) and the ANSWER-ASAP project (TIN2017-85160-C2-1-R) from MINECO, and from Xunta de Galicia (ED431B 2017/01). MA and AS are funded by a Google Focused Research Award.

Appendix A Appendices

For the bilstm-based model, we essentially follow the configuration of the baseline Gómez-Rodríguez and Vilares (2018) for an homogenous comparison. We detail the hyperparameters in Table 7.999Note that the noise sampling is only used for Swedish in the final models based on development set results with and without it.

**Corrigendum to Better, Faster, Stronger Sequence Tagging Constituent Parsers

**

Results after correction

Note: For model selection, we still do not exclude any non-terminal or pre-terminal from the evaluation, while for official comparison on the dev and test sets we now use the collins.prm and spmrl.prm files to parametrize the evalb scripts.

This corrected version contains improved results for the experiments on the ptb, as the collins.prm file excludes from the evaluation some pre-terminals related to punctuation. For the experiments in the spmrl datasets, punctuation is taken into account, but the non-terminals top, s1, root, vroot are stripped off when using the spmrl.prm parameter file. This translates into lower results ($\sim$0.6 points on average), but the tendencies showed in the paper still hold.

With respect to the experiments with the full models, we were relying on the models trained with the auxiliary task that performed the best on the development set. Although differences across auxiliary tasks were in general small; for most of the treebanks the auxiliary task that performed the best with the buggy evaluation still keeps to do so with the corrected one. There are two exceptions where the ranking of the top auxiliary task change by a tiny difference: English (0.02) and Hebrew (0.01). For these models, we re-trained and updated the full models accordingly.