Optimizing Differentiable Relaxations of Coreference Evaluation Metrics

TL;DR

This paper introduces a differentiable relaxation of coreference evaluation metrics, enabling direct gradient-based optimization and improving neural coreference system performance without reinforcement learning.

Contribution

We propose a novel differentiable relaxation of coreference metrics, allowing direct optimization and outperforming methods relying on reinforcement learning.

Findings

Significant performance gains in neural coreference systems

Differentiable relaxation enables direct metric optimization

Outperforms reinforcement learning approaches

Abstract

Coreference evaluation metrics are hard to optimize directly as they are non-differentiable functions, not easily decomposable into elementary decisions. Consequently, most approaches optimize objectives only indirectly related to the end goal, resulting in suboptimal performance. Instead, we propose a differentiable relaxation that lends itself to gradient-based optimisation, thus bypassing the need for reinforcement learning or heuristic modification of cross-entropy. We show that by modifying the training objective of a competitive neural coreference system, we obtain a substantial gain in performance. This suggests that our approach can be regarded as a viable alternative to using reinforcement learning or more computationally expensive imitation learning.

Tables

Table 1. Table 1: Results (F 1 ) on CoNLL 2012 test set. CoNLL is the average of MUC, B 3 , and CEAF e .

	MUC	B3	CEAFm	CEAFe	BLANC	LEA	CoNLL
Wiseman et al. (2015b)	72.60	60.52	-	57.05	-	-	63.39
Wiseman et al. (2016)	73.42	61.50	-	57.70	-	-	64.21
Our proposals
baseline (heuristic loss)	73.22	61.44	65.12	57.74	62.16	57.52	64.13
$L_{ec}$	73.2	61.75	65.77	57.8	63.3	57.89	64.25
\hdashline$L_{\beta =1,B^3}$	73.37	61.94	65.79	58.22	63.19	58.06	64.51
$L_{\beta =\sqrt{1.4},B^3}$	73.48	61.99	65.9	58.36	63.1	58.13	64.61
\hdashline$L_{\beta =1,LEA}$	73.3	61.88	65.69	57.99	63.27	58.03	64.39
$L_{\beta =\sqrt{1.8},LEA}$	73.53	62.04	65.95	58.41	63.09	58.18	64.66
Clark and Manning (2016a)
baseline (heuristic loss)	74.65	63.03	-	58.40	-	-	65.36
REINFORCE	74.48	63.09	-	58.67	-	-	65.41
Reward Rescaling	74.56	63.40	-	59.23	-	-	65.73

Table 2. Table 2: Number of: “false anaphor” (FA, a non-anaphoric mention marked as anaphoric), “false new” (FN, an anaphoric mention marked as non-anaphoric), and “wrong link” (WL, an anaphoric mention is linked to a wrong antecedent) errors on the development set.

	Non-Anaphoric (FA)			Anaphoric (FN + WL)
	Proper	Nominal	Pronom.	Proper	Nominal	Pronom.
baseline	630	714	1051	374 + 190	821 + 238	347 + 779
$L_{ec}$	529	609	904	438 + 182	924 + 220	476 + 740
\hdashline$L_{\beta =1,B^3}$	545	559	883	433 + 172	951 + 192	457 + 761
$L_{\beta =\sqrt{1.4},B^3}$	557	564	926	426 + 178	941 + 194	431 + 766
\hdashline$L_{\beta =1,LEA}$	513	547	843	456 + 170	960 + 191	513 + 740
$L_{\beta =\sqrt{1.8},LEA}$	577	591	1001	416 + 176	919 + 198	358 + 790

Equations

$$\mathsf{s}(a_i=j)=\{\begin{array}{ll}{\mathbf{u}}^T\left[\begin{array}{c}{\mathbf{h}}_a(m_i)\\ {\mathbf{h}}_p(m_i,m_j)\end{array}\right]+u_0& \text{if }j<i\\ {\mathbf{v}}^T{\mathbf{h}}_a(m_i)+v_0& \text{if }j=i\end{array}$$

$${\mathbf{h}}_a(m_i)$$

$${\mathbf{h}}_p(m_i,m_j)$$

$$p(a_i=j)=\frac{\exp \{\mathsf{s}(a_i=j)\}}{{\sum }_{j^{\prime }=1}^i\exp \{\mathsf{s}(a_i=j^{\prime })\}}$$

$$L(\Theta)=-\sum_{i=1}^{n}\log\big{(}\sum_{j\in C(m_{i})}p^{\prime}(a_{i}=j)\big{)}+\lambda||\Theta||_{1},$$

$$\Delta (j,C(m_i))=\{\begin{array}{ll}{\alpha }_1& \text{if }j\ne i\wedge i\in C(m_i)\\ {\alpha }_2& \text{if }j=i\wedge i\notin C(m_i)\\ {\alpha }_3& \text{if }j\ne i\wedge j\notin C(m_i)\\ 0& \text{otherwise}\end{array}$$

$$R_{B^3}$$

$$P_{B^3}$$

$$R_{LEA}$$

$$P_{LEA}$$

$$F_{\beta }=(1+{\beta }^2)\frac{P\times R}{{\beta }^{2}P+R}$$

$$p(m_i\in E_u)=$$

$$\{\begin{array}{ll}{\sum }_{j=u}^{i-1}p(a_i=j)\times p(m_j\in E_u)& \text{if }u<i\\ p(a_i=i)& \text{if }u=i\\ 0& \text{if }u>i\end{array}$$

$$\sum _{u=1}^{i-1}p(m_i\in E_u)$$

$$=\sum _{u=1}^{i-1}\sum _{j=u}^{i-1}p(a_i=j)\times p(m_j\in E_u)$$

$$\sum _{u=1}^{i-1}\sum _{j=1}^{i-1}p(a_i=j)\times p(m_j\in E_u)$$

$$=\sum _{j=1}^{i-1}p(a_i=j)\times \sum _{u=1}^{i-1}p(m_j\in E_u)$$

$$=\sum _{j=1}^{i-1}p(a_i=j)$$

$$\sum _{u=1}^{n}p(m_i\in E_u)=\sum _{j=1}^{i-1}p(a_i=j)+p(a_i=i)=1$$

$$p(m_{u+1}\in E_u)$$

$$\le p(m_u\in E_u)$$

$$p(m_i\in E_u)$$

$$\le \sum _{j=u}^{i-1}p(a_i=j)\times p(m_u\in E_u)$$

$$\le p(m_u\in E_u)\times \sum _{j=1}^{i}p(a_i=j)$$

$$=p(m_u\in E_u)$$

$$L_{ec}(\Theta )=-\sum _{i=1}^n\log p^{\prime }(m_i\in E_{e(m_i)})+\lambda ||\Theta |{|}_1$$

$$\Gamma (u,e(m_i))=$$

$$\{\begin{array}{ll}{\gamma }_1& \text{if }u\ne i\wedge e(m_i)=i\\ {\gamma }_2& \text{if }u=i\wedge e(m_i)\ne i\\ {\gamma }_3& \text{if }u\ne e(m_i)\wedge u\ne i\wedge e(m_i)\ne i\\ 0& \text{otherwise}\end{array}$$

$$|S_u|$$

$$link(S_u)$$

$$p(m_i\in S_u;T)=\frac{\exp \{{\pi }_{i,u}/T\}}{{\sum }_v\exp \{{\pi }_{i,v}/T\}}$$

$$S_u=\{p(m_1\in E_u;T),...,p(m_n\in E_u;T)\}$$

$$|S_u{|}_d$$

$$lin{k}_d(S_u)$$

$$L_{\beta ,B^3}(\Theta ;T)$$

$$L_{\beta ,LEA}(\Theta ;T)$$

Full text

1 Introduction

Coreference resolution is the task of identifying all mentions which refer to the same entity in a document. It has been shown beneficial in many natural language processing (NLP) applications, including question answering Hermann et al. (2015) and information extraction Kehler (1997), and often regarded as a prerequisite to any text understanding task.

Coreference resolution can be regarded as a clustering problem: each cluster corresponds to a single entity and consists of all its mentions in a given text. Consequently, it is natural to evaluate predicted clusters by comparing them with the ones annotated by human experts, and this is exactly what the standard metrics (e.g., MUC, B3, CEAF) do. In contrast, most state-of-the-art systems are optimized to make individual co-reference decisions, and such losses are only indirectly related to the metrics.

One way to deal with this challenge is to optimize directly the non-differentiable metrics using reinforcement learning (RL), for example, relying on the REINFORCE policy gradient algorithm Williams (1992). However, this approach has not been very successful, which, as suggested by Clark and Manning (2016a), is possibly due to the discrepancy between sampling decisions at training time and choosing the highest ranking ones at test time. A more successful alternative is using a ‘roll-out’ stage to associate cost with possible decisions, as in Clark and Manning (2016a), but it is computationally expensive. Imitation learning Ma et al. (2014b); Clark and Manning (2015), though also exploiting metrics, requires access to an expert policy, with exact policies not directly computable for the metrics of interest.

In this work, we aim at combining the best of both worlds by proposing a simple method that can turn popular coreference evaluation metrics into differentiable functions of model parameters. As we show, this function can be computed recursively using scores of individual local decisions, resulting in a simple and efficient estimation procedure. The key idea is to replace non-differentiable indicator functions (e.g. the member function $\mathbb{I}(m\in S)$) with the corresponding posterior probabilities ($p(m\in S)$) computed by the model. Consequently, non-differentiable functions used within the metrics (e.g. the set size function $|S|=\sum_{m}\mathbb{I}(m\in S)$) become differentiable ($|S|_{c}=\sum_{m}p(m\in S)$). Though we assume that the scores of the underlying statistical model can be used to define a probability model, we show that this is not a serious limitation. Specifically, as a baseline we use a probabilistic version of the neural mention-ranking model of Wiseman et al. (2015b), which on its own outperforms the original one and achieves similar performance to its global version Wiseman et al. (2016). Importantly when we use the introduced differentiable relaxations in training, we observe a substantial gain in performance over our probabilistic baseline. Interestingly, the absolute improvement (+0.52) is higher than the one reported in Clark and Manning (2016a) using RL (+0.05) and the one using reward rescaling111Reward rescaling is a technique that computes error values for a heuristic loss function based on the reward difference between the best decision according to the current model and the decision leading to the highest metric score. (+0.37). This suggests that our method provides a viable alternative to using RL and reward rescaling.

The outline of our paper is as follows: we introduce our neural resolver baseline and the B3 and LEA metrics in Section 2. Our method to turn a mention ranking resolver into an entity-centric resolver is presented in Section 3, and the proposed differentiable relaxations in Section 4. Section 5 shows our experimental results.

2 Background

2.1 Neural mention ranking

In this section we introduce neural mention ranking, the framework which underpins current state-of-the-art models Clark and Manning (2016a). Specifically, we consider a probabilistic version of the method proposed by Wiseman et al. (2015b). In experiments we will use it as our baseline.

Let $(m_{1},m_{2},..,m_{n})$ be the list of mentions in a document. For each mention $m_{i}$, let $a_{i}\in\{1,...,i\}$ be the index of the mention that $m_{i}$ is coreferent with (if $a_{i}=i$, $m_{i}$ is the first mention of some entity appearing in the document). As standard in coreference resolution literature, we will refer to $m_{a_{i}}$ as an antecedent of $m_{i}$.222This slightly deviates from the definition of antecedents in linguistics Crystal (1997). Then, in mention ranking the goal is to score antecedents of a mention higher than any other mentions, i.e., if $\mathsf{s}$ is the scoring function, we require $\mathsf{s}(a_{i}=j)>\mathsf{s}(a_{i}=k)$ for all $j,k$ such that $m_{i}$ and $m_{j}$ are coreferent but $m_{i}$ and $m_{k}$ are not.

Let $\phi_{a}(m_{i})\in\mathbb{R}^{d_{a}}$ and $\phi_{p}(m_{i},m_{j})\in\mathbb{R}^{d_{p}}$ be respectively features of $m_{i}$ and features of pair $(m_{i},m_{j})$. The scoring function is defined by:

[TABLE]

where

[TABLE]

and $\mathbf{u},\mathbf{v},\mathbf{W}_{a},\mathbf{W}_{p},\mathbf{b}_{a},\mathbf{b}_{p}$ are real vectors and matrices with proper dimensions, $u_{0},v_{0}$ are real scalars.

Unlike Wiseman et al. (2015b), where the max-margin loss is used, we define a probabilistic model. The probability333For the sake of readability, we do not explicitly mark in our notation that all the probabilities are conditioned on the document (e.g., the mentions) and dependent on model parameters. that $m_{i}$ and $m_{j}$ are coreferent is given by

[TABLE]

Following Durrett and Klein (2013) we use the following softmax-margin Gimpel and Smith (2010) loss function:

[TABLE]

where $\Theta$ are model parameters, $C(m_{i})$ is the set of the indices of correct antecedents of $m_{i}$, and $p^{\prime}(a_{i}=j)\propto p(a_{i}=j)e^{\Delta(j,C(m_{i}))}$. $\Delta$ is a cost function used to manipulate the contribution of different error types to the loss function:

[TABLE]

The error types are “false anaphor”, “false new”, “wrong link”, and “no mistake”, respectively. In our experiments, we borrow their values from Durrett and Klein (2013): $(\alpha_{1},\alpha_{2},\alpha_{3})=(0.1,3,1)$. In the subsequent discussion, we refer to the loss as mention-ranking heuristic cross entropy.

2.2 Evaluation Metrics

We use five most popular metrics444All are implemented in Pradhan et al. (2014), https://github.com/conll/reference-coreference-scorers.,

• •

MUC Vilain et al. (1995),

• •

B3 Bagga and Baldwin (1998),

• •

CEAFm, CEAFe Luo (2005),

• •

BLANC Luo et al. (2014),

• •

LEA Moosavi and Strube (2016).

for evaluation. However, because MUC is the least discriminative metric Moosavi and Strube (2016), whereas CEAF is slow to compute, out of the five most popular metrics we incorporate into our loss only B3. In addition, we integrate LEA, as it has been shown to provide a good balance between discriminativity and interpretability.

Let $G=\{G_{1},G_{2},...,G_{N}\}$ and $S=\{S_{1},S_{2},...,S_{M}\}$ be the gold-standard entity set and an entity set given by a resolver. Recall that an entity is a set of mentions. The recall and precision of the B3 metric is computed by:

[TABLE]

The LEA metric is computed as:

[TABLE]

where $link(E)=|E|\times(|E|-1)/2$ is the number of coreference links in entity $E$. $F_{\beta}$, for both metrics, is defined by:

[TABLE]

$\beta=1$ is used in the standard evaluation.

3 From mention ranking to entity centricity

Mention-ranking resolvers do not explicitly provide information about entities/clusters which is required by B3 and LEA. We therefore propose a simple solution that can turn a mention-ranking resolver into an entity-centric one.

First note that in a document containing $n$ mentions, there are $n$ potential entities $E_{1},E_{2},...,E_{n}$ where $E_{i}$ has $m_{i}$ as the first mention. Let $p(m_{i}\in E_{u})$ be the probability that mention $m_{i}$ corresponds to entity $E_{u}$. We now show that it can be computed recursively based on $p(a_{i}=j)$ as follows:

[TABLE]

In other words, if $u<i$, we consider all possible $m_{j}$ with which $m_{i}$ can be coreferent, and which can correspond to entity $E_{u}$. If $u=i$, the link to be considered is the $m_{i}$’s self-link. And, if $u>i$, the probability is zero, as it is impossible for $m_{i}$ to be assigned to an entity introduced only later. See Figure 1 for extra information.

We now turn to two crucial questions about this formula:

• •

Is $p(m_{i}\in\bullet)$ a valid probability distribution?

• •

*Is it possible for a mention $m_{u}$ to be mostly anaphoric (i.e. $p(m_{u}\in E_{u})$ is low) but for the corresponding cluster $E_{u}$ to be highly probable (i.e. $p(m_{i}\in E_{u})$ is high for some $i$)? *

The first question is answered in Proposition 1. The second question is important because, intuitively, when a mention $m_{u}$ is anaphoric, the potential entity $E_{u}$ does not exist. We will show that the answer is “No” by proving in Proposition 2 that the probability that $m_{u}$ is anaphoric is always higher than any probability that $m_{i}$, $i>u$ refers to $E_{u}$.

Proposition 1**.**

$p(m_{i}\in\bullet)$* is a valid probability distribution, i.e., $\sum_{u=1}^{n}p(m_{i}\in E_{u})=1$, for all $i=1,...,n$.*

Proof.

We prove this proposition by induction.

Basis: it is obvious that $\sum_{u=1}^{n}p(m_{1}\in E_{u})=p(a_{1}=1)=1$.

Assume that $\sum_{u=1}^{n}p(m_{j}\in E_{u})=1$ for all $j<i$. Then,

[TABLE]

Because $p(m_{j}\in E_{u})=0$ for all $j<u$, this expression is equal to

[TABLE]

Therefore,

[TABLE]

(according to Equation 1).

∎

Proposition 2**.**

$p(m_{i}\in E_{u})\leq p(m_{u}\in E_{u})$* for all $i>u$.*

Proof.

We prove this proposition by induction.

Basis: for $i=u+1$,

[TABLE]

Assume that $p(m_{j}\in E_{u})\leq p(m_{u}\in E_{u})$ for all $j\geq u$ and $j<i$. Then

[TABLE]

∎

3.1 Entity-centric heuristic cross entropy loss

Having $p(m_{i}\in E_{u})$ computed, we can consider coreference resolution as a multiclass prediction problem. An entity-centric heuristic cross entropy loss is thus given below:

[TABLE]

where $E_{e(m_{i})}$ is the correct entity that $m_{i}$ belongs to, $p^{\prime}(m_{i}\in E_{u})\propto p(m_{i}\in E_{u})e^{\Gamma(u,e(m_{i}))}$. Similar to $\Delta$ in the mention-ranking heuristic loss in Section 2.1, $\Gamma$ is a cost function used to manipulate the contribution of the four different error types (“false anaphor”, “false new”, “wrong link”, and “no mistake”):

[TABLE]

4 From non-differentiable metrics to differentiable losses

There are two functions used in computing B3 and LEA: the set size function $|.|$ and the link function $link(.)$. Because both of them are non-differentiable, the two metrics are non-differentiable. We thus need to make these two functions differentiable.

There are two remarks. Firstly, both functions can be computed using the indicator function $\mathbb{I}(m_{i}\in S_{u})$:

[TABLE]

Secondly, given $\pi_{i,u}=\log p(m_{i}\in S_{u})$, the indicator function $\mathbb{I}(m_{i}\in S_{u^{*}})$, $u^{*}=\operatorname*{arg\,max}_{u}p(m_{i}\in S_{u})$ is the converging point of the following softmax as $T\rightarrow 0$ (see Figure 2):

[TABLE]

where $T$ is called temperature Kirkpatrick et al. (1983).

Therefore, we propose to represent each $S_{u}$ as a soft-cluster:

[TABLE]

where, as defined in Section 3, $E_{u}$ is the potential entity that has $m_{u}$ as the first mention. Replacing the indicator function $\mathbb{I}(m_{i}\in S_{u})$ by the probability distribution $p(m_{i}\in E_{u};T)$, we then have a differentiable version for the set size function and the link function:

[TABLE]

$|G_{v}\cap S_{u}|_{d}$ and $link_{d}(G_{v}\cap S_{u})$ are computed similarly with the constraint that only mentions in $G_{v}$ are taken into account. Plugging these functions into precision and recall of B3 and LEA in Section 2.2, we obtain differentiable $\hat{F}_{\beta,B^{3}}$ and $\hat{F}_{\beta,LEA}$, which are then used in two loss functions:

[TABLE]

where $\lambda$ is the hyper-parameter of the $L_{1}$ regularization terms.

It is worth noting that, as $T\rightarrow 0$, $\hat{F}_{\beta,B^{3}}\rightarrow F_{\beta,B^{3}}$ and $\hat{F}_{\beta,LEA}\rightarrow F_{\beta,LEA}$.555We can easily prove this using the algebraic limit theorem. Therefore, when training a model with the proposed losses, we can start at a high temperature (e.g., $T=1$) and anneal to a small but non-zero temperature. However, in our experiments we fix $T=1$. Annealing is left for future work.

5 Experiments

We now demonstrate how to use the proposed differentiable B3 and LEA to train a coreference resolver. The source code and trained models are available at https://github.com/lephong/diffmetric_coref.

Setup

We run experiments on the English portion of CoNLL 2012 data Pradhan et al. (2012) which consists of 3,492 documents in various domains and formats. The split provided in the CoNLL 2012 shared task is used. In all our resolvers, we use not the original features of Wiseman et al. (2015b) but their slight modification described in Wiseman et al. (2016) (section 6.1).666https://github.com/swiseman/nn_coref/

Resolvers

We build following baseline and three resolvers:

• •

baseline: the resolver presented in Section 2.1. We use the identical configuration as in Wiseman et al. (2016): $\mathbf{W}_{a}\in\mathbb{R}^{200\times d_{a}}$, $\mathbf{W}_{p}\in\mathbb{R}^{700\times d_{p}}$, $\lambda=10^{-6}$ (where $d_{a},d_{p}$ are respectively the numbers of mention features and pair-wise features). We also employ their pretraining methodology.

• •

$L_{ec}$: the resolver using the entity-centric cross entropy loss introduced in Section 3.1. We set $(\gamma_{1},\gamma_{2},\gamma_{3})=(\alpha_{1},\alpha_{2},\alpha_{3})=(0.1,3,1)$.

• •

$L_{\beta,B^{3}}$ and $L_{\beta,LEA}$: the resolvers using the losses proposed in Section 4. $\beta$ is tuned on the development set by trying each value in $\{\sqrt{0.8},1,\sqrt{1.2},\sqrt{1.4},\sqrt{1.6},\sqrt{1.8},1.5,2\}$.

To train these resolvers we use AdaGrad Duchi et al. (2011) to minimize their loss functions with the learning rate tuned on the development set and with one-document mini-batches. Note that we use the baseline as the initialization point to train the other three resolvers.

5.1 Results

We firstly compare our resolvers against Wiseman et al. (2015b) and Wiseman et al. (2016). Results are shown in the first half of Table 1. Our baseline surpasses Wiseman et al. (2015b). It is likely due to using features from Wiseman et al. (2016). Using the entity-centric heuristic cross entropy loss and the relaxations are clearly beneficial: $L_{ec}$ is slightly better than our baseline and on par with the global model of Wiseman et al. (2016). $L_{\beta=1,B^{3}},L_{\beta=1,LEA}$ outperform the baseline, the global model of Wiseman et al. (2016), and $L_{ec}$. However, the best values of $\beta$ are $\sqrt{1.4}$, $\sqrt{1.8}$ respectively for $L_{\beta,B^{3}}$, and $L_{\beta,LEA}$. Among these resolvers, $L_{\beta=\sqrt{1.8},LEA}$ achieves the highest F1 scores across all the metrics except BLANC.

When comparing to Clark and Manning (2016a) (the second half of Table 1), we can see that the absolute improvement over the baselines (i.e. ‘heuristic loss’ for them and the heuristic cross entropy loss for us) is higher than that of reward rescaling but with much shorter training time: $+0.37$ (7 days777As reported in https://github.com/clarkkev/deep-coref) and $+0.52$ (15 hours) on the CoNLL metric for Clark and Manning (2016a) and ours, respectively. It is worth noting that our absolute scores are weaker than these of Clark and Manning (2016a), as they build on top of a similar but stronger mention-ranking baseline, which employs deeper neural networks and requires a much larger number of epochs to train (300 epochs, including pretraining). For the purpose of illustrating the proposed losses, we started with a simpler model by Wiseman et al. (2015b) which requires a much smaller number of epochs, thus faster, to train (20 epochs, including pretraining).

5.2 Analysis

Table 2 shows the breakdown of errors made by the baseline and our resolvers on the development set. The proposed resolvers make fewer “false anaphor” and “wrong link” errors but more “false new” errors compared to the baseline. This suggests that loss optimization prevents over-clustering, driving the precision up: when antecedents are difficult to detect, the self-link (i.e., $a_{i}=i$) is chosen. When $\beta$ increases, they make more “false anaphor” and “wrong link” errors but less “false new” errors.

In Figure 3(a) the baseline, but not $L_{\beta=1,B^{3}}$ nor $L_{\beta=\sqrt{1.4},B^{3}}$, mistakenly links 17[it] with 13[the virus]. Under-clustering, on the other hand, is a problem for our resolvers with $\beta=1$: in example (b), $L_{\beta=1,B^{3}}$ missed 165[We]. This behaviour results in a reduced recall but the recall is not damaged severely, as we still obtain a better $F_{1}$ score. We conjecture that this behaviour is a consequence of using the $F_{1}$ score in the objective, and, if undesirable, Fβ with $\beta>1$ can be used instead. For instance, also in Figure 3, $L_{\beta=\sqrt{1.4},B^{3}}$ correctly detects 17[it] as non-anaphoric and links 165[We] with 157[our].

Figure 4 shows recall, precision, F1 (average of MUC, B3, CEAFe), on the development set when training with $L_{\beta,B^{3}}$ and $L_{\beta,LEA}$. As expected, higher values of $\beta$ yield lower precisions but higher recalls. In contrast, F1 increases until reaching the highest point when $\beta=\sqrt{1.4}\approx 1.18$ for $L_{\beta,B^{3}}$ ($\beta=\sqrt{1.8}\approx 1.34$ for $L_{\beta,LEA}$), it then decreases gradually.

5.3 Discussion

Because the resolvers are evaluated on F1 score metrics, it should be that $L_{\beta,B^{3}}$ and $L_{\beta,LEA}$ perform the best with $\beta=1$. Figure 4 and Table 1 however do not confirm that: $\beta$ should be set with values a little bit larger than 1. There are two hypotheses. First, the statistical difference between the training set and the development set leads to the case that the optimal $\beta$ on one set can be sub-optimal on the other set. Second, in our experiments we fix $T=1$, meaning that the relaxations might not be close to the true evaluation metrics enough. Our future work, to confirm/reject this, is to use annealing, i.e., gradually decreasing $T$ down to (but larger than) 0.

Table 1 shows that the difference between $L_{\beta,B^{3}}$ and $L_{\beta,LEA}$ in terms of accuracy is not substantial (although the latter is slightly better than the former). However, one should expect that $L_{\beta,B^{3}}$ would outperform $L_{\beta,LEA}$ on B3 metric while it would be the other way around on LEA metric. It turns out that, B3 and LEA behave quite similarly in non-extreme cases. We can see that in Figure 2, 4, 5, 6, 7 in Moosavi and Strube (2016).

6 Related work

Mention ranking and entity centricity are two main streams in the coreference resolution literature. Mention ranking Denis and Baldridge (2007); Durrett and Klein (2013); Martschat and Strube (2015); Wiseman et al. (2015a) considers local and independent decisions when choosing a correct antecedent for a mention. This approach is computationally efficient and currently dominant with state-of-the-art performance Wiseman et al. (2016); Clark and Manning (2016a). Wiseman et al. (2015b) propose to use simple neural networks to compute mention ranking scores and to use a heuristic loss to train the model. Wiseman et al. (2016) extend this by employing LSTMs to compute mention-chain representations which are then used to compute ranking scores. They call these representations global features. Clark and Manning (2016a) build a similar resolver as in Wiseman et al. (2015b) but much stronger thanks to deeper neural networks and “better mention detection, more effective, hyperparameters, and more epochs of training”. Furthermore, using reward rescaling they achieve the best performance in the literature on the English and Chinese portions of the CoNLL 2012 dataset. Our work is built upon mention ranking by turning a mention-ranking model into an entity-centric one. It is worth noting that although we use the model proposed by Wiseman et al. (2015b), any mention-ranking models can be employed.

Entity centricity Wellner and McCallum (2003); Poon and Domingos (2008); Haghighi and Klein (2010); Ma et al. (2014a); Clark and Manning (2016b), on the other hand, incorporates entity-level information to solve the problem. The approach can be top-down as in Haghighi and Klein (2010) where they propose a generative model. It can also be bottom-up by merging smaller clusters into bigger ones as in Clark and Manning (2016b). The method proposed by Ma et al. (2014a) greedily and incrementally adds mentions to previously built clusters using a prune-and-score technique. Importantly, employing imitation learning these two methods can optimize the resolvers directly on evaluation metrics. Our work is similar to Ma et al. (2014a) in the sense that our resolvers incrementally add mentions to previously built clusters. However, different from both Ma et al. (2014a); Clark and Manning (2016b), our resolvers do not use any discrete decisions (e.g., merge operations). Instead, they seamlessly compute the probability that a mention refers to an entity from mention-ranking probabilities, and are optimized on differentiable relaxations of evaluation metrics.

Using differentiable relaxations of evaluation metrics as in our work is related to a line of research in reinforcement learning where a non-differentiable action-value function is replaced by a differentiable critic Sutton et al. (1999); Silver et al. (2014). The critic is trained so that it is as close to the true action-value function as possible. This technique is applied to machine translation Gu et al. (2017) where evaluation metrics (e.g., BLUE) are non-differentiable. A disadvantage of using critics is that there is no guarantee that the critic converges to the true evaluation metric given finite training data. In contrast, our differentiable relaxations do not need to train, and the convergence is guaranteed as $T\rightarrow 0$.

7 Conclusions

We have proposed

• •

a method for turning any mention-ranking resolver into an entity-centric one by using a recursive formula to combine scores of individual local decisions, and

• •

differentiable relaxations for two coreference evaluation metrics, B3 and LEA.

Experimental results show that our approach outperforms the resolver by Wiseman et al. (2016), and gains a higher improvement over the baseline than that of Clark and Manning (2016a) but with much shorter training time.

Acknowledgments

We would like to thank Raquel Fernández, Wilker Aziz, Nafise Sadat Moosavi, and anonymous reviewers for their suggestions and comments. The project was supported by the European Research Council (ERC StG BroadSem 678254), the Dutch National Science Foundation (NWO VIDI 639.022.518) and an Amazon Web Services (AWS) grant.