Interpretable Neural Predictions with Differentiable Binary Variables

TL;DR

This paper introduces a novel neural network approach that jointly learns to select interpretable rationales from text and classify based on them, using a differentiable method that avoids complex gradient estimators.

Contribution

It proposes a mixed discrete-continuous latent model enabling gradient-based training of binary rationale selectors with explicit control over selection sparsity.

Findings

Competitive performance on rationale extraction tasks

Effective gradient-based training of binary selectors

Potential applications in attention mechanisms

Abstract

The success of neural networks comes hand in hand with a desire for more interpretability. We focus on text classifiers and make them more interpretable by having them provide a justification, a rationale, for their predictions. We approach this problem by jointly training two neural network models: a latent model that selects a rationale (i.e. a short and informative part of the input text), and a classifier that learns from the words in the rationale alone. Previous work proposed to assign binary latent masks to input positions and to promote short selections via sparsity-inducing penalties such as L0 regularisation. We propose a latent model that mixes discrete and continuous behaviour allowing at the same time for binary selections and gradient-based training without REINFORCE. In our formulation, we can tractably compute the expected value of penalties such as L0, which allows us to directly optimise the model towards a pre-specified text selection rate. We show that our approach is competitive with previous work on rationale extraction, and explore further uses in attention mechanisms.

Tables

Table 1. Table 1: MSE on the BeerAdvocate test set.

SVM (Lei et al., 2016)	0.0154
BiLSTM (Lei et al., 2016)	0.0094
BiRCNN (Lei et al., 2016)	0.0087
BiLSTM (ours)	0.0089
BiRCNN (ours)	0.0088

Table 2. Table 2: Precision (% of selected words that was also annotated as the gold rationale) and selected (% of words not zeroed out) per aspect. In the attention baseline, the top 13% (7%) of words with highest attention weights are used for classification. Models were selected based on validation loss.

Method	Look		Smell		Palate
	% Precision	% Selected	% Precision	% Selected	% Precision	% Selected
Attention (Lei et al.)	80.6	13	88.4	7	65.3	7
Bernoulli (Lei et al.)	96.3	14	95.1	7	80.2	7
Bernoulli (reimpl.)	94.8	13	95.1	7	80.5	7
HardKuma	98.1	13	96.8	7	89.8	7

Table 3. Table 3: SNLI results (accuracy).

Model	Dev	Test
LSTM (Bowman et al., 2016)	–	80.6
DA (Parikh et al., 2016)	–	86.3
DA (reimplementation)	86.9	86.5
DA with HardKuma attention	86.0	85.5

Table 4. Table 4: Beer hyperparameters.

Optimizer	Adam
Learning rate	0.0004
Word embeddings	200D (Wiki, fixed)
Hidden size	200
Batch size	256
Dropout	0.1, 0.2
Weight decay	$1\ast {10}^{-6}$
Cell	RCNN

Table 5. Table 5: SST hyperparameters.

Optimizer	Adam
Learning rate	0.0002
Word embeddings	300D Glove (fixed)
Hidden size	150
Batch size	25
Dropout	0.5
Weight decay	$1\ast {10}^{-6}$
Cell	LSTM

Table 6. Table 6: SNLI hyperparameters.

Optimizer	Adam
Learning rate	0.0001
Word embeddings	300D (Glove, fixed)
Hidden size	200
Batch size	64
Dropout	0.2

Equations

$$Y|x\sim \mathrm{Cat}(f(x;\theta )),$$

$$Z_i|x$$

$$Y|x,z$$

$$\log P(y|x)=\log {\mathbb{E}}_{P(z|x,\varphi )}\left[P(y|x,z,\theta )\right]$$

$$\stackrel{\text{JI}}{\ge }{\mathbb{E}}_{P(z|x,\varphi )}\left[\log P(y|x,z,\theta )\right]=\mathcal{E}(\varphi ,\theta ).$$

$$\underset{\varphi ,\theta }{\min }-\mathcal{E}(\varphi ,\theta )+{\lambda }_0\underset{L_0(z)}{\underbrace{\sum _{i=1}^n{z}_i}}+{\lambda }_1\underset{\text{fused lasso}}{\underbrace{\sum _{i=1}^{n-1}|z_i-z_{i+1}|}}$$

$$F_K^{-1}(u;a,b)={\left(1-(1-u{)}^{\text{nicefrac}1b}\right)}^{\text{nicefrac}1a},$$

$$F_K^{-1}(U;\alpha ,\beta )\sim \mathrm{Kuma}(\alpha ,\beta )$$

$$F_T(t;a,b,l,r)=F_K(\text{nicefrac}(t-l)(r-l);a,b).$$

$$\mathbb{P}(H=0)=F_K\left(\frac{-l}{r-l};a,b\right).$$

$$\mathbb{P}(H=1)=1-F_K\left(\frac{1-l}{r-l};a,b\right).$$

$$k$$

$$t$$

$$h$$

$${\mathbb{E}}_{p(z|x)}\left[L_0(z)\right]\stackrel{\text{ind}}{=}\sum _{i=1}^n{\mathbb{E}}_{p(z_i|x)}\left[\mathbb{I}[z_i\ne 0]\right]$$

$$=\sum _{i=1}^{n}1-\mathbb{P}(Z_i=0),$$

$${\mathbb{E}}_{p(z|x)}\left[\sum _{i=1}^{n-1}\mathbb{I}[z_i=0,z_{i+1}\ne 0]\right]$$

$$+{\mathbb{E}}_{p(z|x)}\left[\sum _{i=1}^{n-1}\mathbb{I}[z_i\ne 0,z_{i+1}=0]\right]$$

$$\stackrel{\text{ind}}{=}\sum _{i=1}^{n-1}\mathbb{P}(Z_i=0)(1-\mathbb{P}(Z_{i+1}=0))$$

$$+(1-\mathbb{P}(Z_i=0))\mathbb{P}(Z_{i+1}=0).$$

$$\underset{\lambda \in \mathbb{R}}{\max }\underset{\varphi ,\theta }{\min }-\mathcal{E}(\varphi ,\theta )+{\lambda }^{\top }(R(\varphi )-r)$$

$$Z_i$$

$$Y|x,z$$

$$\begin{array}{rl}{\mathbf{e}}_i& =\mathrm{emb}(x_i)\\ {\mathbf{h}}_1^n& =\mathrm{birnn}({\mathbf{e}}_1^n;{\varphi }_r)\\ u_i& \sim \mathcal{U}(0,1)\end{array}\begin{array}{rl}{a}_i& =f_a({\mathbf{h}}_i;{\varphi }_a)\\ b_i& =f_b({\mathbf{h}}_i;{\varphi }_b)\\ z_i& =s(u_i;a_i,b_i,l,r)\end{array}$$

$${\mathbf{e}}_i$$

$${\mathbf{h}}_i^{(\text{fwd})}$$

$${\mathbf{h}}_i^{(\text{bwd})}$$

$$\mathbf{o}$$

$$\mathcal{E}(\varphi ,\theta )={\mathbb{E}}_{\mathcal{U}(0,I)}\left[\log P(y|x,s_{\varphi }(u,x),\theta )\right]$$

$$a_i$$

$$b_i$$

$${\mathbf{s}}_i$$

$$f_K(k;a,b)=ab{k}^{a-1}(1-k^a{)}^{b-1},$$

$$F_K(k;a,b)$$

$$=1-(1-k^a{)}^b,$$

$$F_K^{-1}(u;a,b)={\left(1-(1-u{)}^{\text{nicefrac}1b}\right)}^{\text{nicefrac}1a}.$$

$$t=l+(r-l)k$$

$$f_T(t;a,b,l,r)$$

Full text

1 Introduction

Neural networks are bringing incredible performance gains on text classification tasks (Howard and Ruder, 2018; Peters et al., 2018; Devlin et al., 2019). However, this power comes hand in hand with a desire for more interpretability, even though its definition may differ (Lipton, 2016). While it is useful to obtain high classification accuracy, with more data available than ever before it also becomes increasingly important to justify predictions. Imagine having to classify a large collection of documents, while verifying that the classifications make sense. It would be extremely time-consuming to read each document to evaluate the results. Moreover, if we do not know why a prediction was made, we do not know if we can trust it.

What if the model could provide us the most important parts of the document, as a justification for its prediction? That is exactly the focus of this paper. We use a setting that was pioneered by Lei et al. (2016). A rationale is defined to be a short yet sufficient part of the input text; short so that it makes clear what is most important, and sufficient so that a correct prediction can be made from the rationale alone. One neural network learns to extract the rationale, while another neural network, with separate parameters, learns to make a prediction from just the rationale. Lei et al. model this by assigning a binary Bernoulli variable to each input word. The rationale then consists of all the words for which a 1 was sampled. Because gradients do not flow through discrete samples, the rationale extractor is optimized using REINFORCE (Williams, 1992). An $L_{0}$ regularizer is used to make sure the rationale is short.

We propose an alternative to purely discrete selectors for which gradient estimation is possible without REINFORCE, instead relying on a reparameterization of a random variable that exhibits both continuous and discrete behavior (Louizos et al., 2017). To promote compact rationales, we employ a relaxed form of $L_{0}$ regularization (Louizos et al., 2017), penalizing the objective as a function of the expected proportion of selected text. We also propose the use of Lagrangian relaxation to target a specific rate of selected input text.

Our contributions are summarized as follows:111Code available at https://github.com/bastings/interpretable_predictions.

we present a differentiable approach to extractive rationales (§2) including an objective that allows for specifying how much text is to be extracted (§4); 2. 2.

we introduce HardKuma (§3), which gives support to binary outcomes and allows for reparameterized gradient estimates; 3. 3.

we empirically show that our approach is competitive with previous work and that HardKuma has further applications, e.g. in attention mechanisms. (§6).

2 Latent Rationale

We are interested in making NN-based text classifiers interpretable by (i) uncovering which parts of the input text contribute features for classification, and (ii) basing decisions on only a fraction of the input text (a rationale). Lei et al. (2016) approached (i) by inducing binary latent selectors that control which input positions are available to an NN encoder that learns features for classification/regression, and (ii) by regularising their architectures using sparsity-inducing penalties on latent assignments. In this section we put their approach under a probabilistic light, and this will then more naturally lead to our proposed method.

In text classification, an input $x$ is mapped to a distribution over target labels:

[TABLE]

where we have a neural network architecture $f(\cdot;\theta)$ parameterize the model—$\theta$ collectively denotes the parameters of the NN layers in $f$. That is, an NN maps from data space (e.g. sentences, short paragraphs, or premise-hypothesis pairs) to the categorical parameter space (i.e. a vector of class probabilities). For the sake of concreteness, consider the input a sequence $x=\langle x_{1},\ldots,x_{n}\rangle$. A target $y$ is typically a categorical outcome, such as a sentiment class or an entailment decision, but with an appropriate choice of likelihood it could also be a numerical score (continuous or integer).

Lei et al. (2016) augment this model with a collection of latent variables which we denote by $z=\langle z_{1},\ldots,z_{n}\rangle$. These variables are responsible for regulating which portions of the input $x$ contribute with predictors (i.e. features) to the classifier. The model formulation changes as follows:

[TABLE]

where an NN $g(\cdot;\phi)$ predicts a sequence of $n$ Bernoulli parameters—one per latent variable—and the classifier is modified such that $z_{i}$ indicates whether or not $x_{i}$ is available for encoding. We can think of the sequence $z$ as a binary gating mechanism used to select a rationale, which with some abuse of notation we denote by $x\odot z$. Figure 1 illustrates the approach.

Parameter estimation for this model can be done by maximizing a lower bound $\mathcal{E}(\phi,\theta)$ on the log-likelihood of the data derived by application of Jensen’s inequality:222This can be seen as variational inference (Jordan et al., 1999) where we perform approximate inference using a data-dependent prior $P(z|x,\phi)$.

[TABLE]

These latent rationales approach the first objective, namely, uncovering which parts of the input text contribute towards a decision. However note that an NN controls the Bernoulli parameters, thus nothing prevents this NN from selecting the whole of the input, thus defaulting to a standard text classifier. To promote compact rationales, Lei et al. (2016) impose sparsity-inducing penalties on latent selectors. They penalise for the total number of selected words, $L_{0}$ in (4), as well as, for the total number of transitions, fused lasso in (4), and approach the following optimization problem

[TABLE]

via gradient-based optimisation, where $\lambda_{0}$ and $\lambda_{1}$ are fixed hyperparameters. The objective is however intractable to compute, the lowerbound, in particular, requires marginalization of $O(2^{n})$ binary sequences. For that reason, Lei et al. sample latent assignments and work with gradient estimates using REINFORCE (Williams, 1992).

The key ingredients are, therefore, binary latent variables and sparsity-inducing regularization, and therefore the solution is marked by non-differentiability. We propose to replace Bernoulli variables by rectified continuous random variables (Socci et al., 1998), for they exhibit both discrete and continuous behaviour. Moreover, they are amenable to reparameterization in terms of a fixed random source (Kingma and Welling, 2014), in which case gradient estimation is possible without REINFORCE. Following Louizos et al. (2017), we exploit one such distribution to relax $L_{0}$ regularization and thus promote compact rationales with a differentiable objective. In section 3, we introduce this distribution and present its properties. In section 4, we employ a Lagrangian relaxation to automatically target a pre-specified selection rate. And finally, in section 5 we present an example for sentiment classification.

3 Hard Kumaraswamy Distribution

Key to our model is a novel distribution that exhibits both continuous and discrete behaviour, in this section we introduce it. With non-negligible probability, samples from this distribution evaluate to exactly [math] or exactly $1$. In a nutshell: i) we start from a distribution over the open interval $(0,1)$ (see dashed curve in Figure 2); ii) we then stretch its support from $l<0$ to $r>1$ in order to include $\{0\}$ and $\{1\}$ (see solid curve in Figure 2); finally, iii) we collapse the probability mass over the interval $(l,0]$ to $\{0\}$, and similarly, the probability mass over the interval $[1,r)$ to $\{1\}$ (shaded areas in Figure 2). This stretch-and-rectify technique was proposed by Louizos et al. (2017), who rectified samples from the BinaryConcrete (or GumbelSoftmax) distribution (Maddison et al., 2017; Jang et al., 2017). We adapted their technique to the Kumaraswamy distribution motivated by its close resemblance to a Beta distribution, for which we have stronger intuitions (for example, its two shape parameters transit rather naturally from unimodal to bimodal configurations of the distribution). In the following, we introduce this new distribution formally.333We use uppercase letters for random variables (e.g. $K$, $T$, and $H$) and lowercase for assignments (e.g. $k$, $t$, $h$). For a random variable $K$, $f_{K}(k;\alpha)$ is the probability density function (pdf), conditioned on parameters $\alpha$, and $F_{K}(k;\alpha)$ is the cumulative distribution function (cdf).

3.1 Kumaraswamy distribution

The Kumaraswamy distribution (Kumaraswamy, 1980) is a two-parameters distribution over the open interval $(0,1)$, we denote a Kumaraswamy-distributed variable by $K\sim\operatorname{Kuma}(a,b)$, where $a\in\mathbb{R}_{>0}$ and $b\in\mathbb{R}_{>0}$ control the distribution’s shape. The dashed curve in Figure 2 illustrates the density of $\operatorname{Kuma}(0.5,0.5)$. For more details including its pdf and cdf, consult Appendix A.

The Kumaraswamy is a close relative of the Beta distribution, though not itself an exponential family, with a simple cdf whose inverse

[TABLE]

for $u\in[0,1]$, can be used to obtain samples

[TABLE]

by transformation of a uniform random source $U\sim\mathcal{U}(0,1)$. We can use this fact to reparameterize expectations (Nalisnick and Smyth, 2016).

3.2 Rectified Kumaraswamy

We stretch the support of the Kumaraswamy distribution to include [math] and $1$. The resulting variable $T\sim\operatorname{Kuma}(a,b,l,r)$ takes on values in the open interval $(l,r)$ where $l<0$ and $r>1$, with cdf

[TABLE]

We now define a rectified random variable, denoted by $H\sim\operatorname{HardKuma}(a,b,l,r)$, by passing a sample $T\sim\operatorname{Kuma}(a,b,l,r)$ through a hard-sigmoid, i.e. $h=\min(1,\max(0,t))$. The resulting variable is defined over the closed interval $[0,1]$. Note that while there is [math] probability of sampling $t=0$, sampling $h=0$ corresponds to sampling any $t\in(l,0]$, a set whose mass under $\operatorname{Kuma}(t|a,b,l,r)$ is available in closed form:

[TABLE]

That is because all negative values of $t$ are deterministically mapped to zero. Similarly, samples $t\in[1,r)$ are all deterministically mapped to $h=1$, whose total mass amounts to

[TABLE]

See Figure 2 for an illustration, and Appendix A for the complete derivations.

3.3 Reparameterization and gradients

Because this rectified variable is built upon a Kumaraswamy, it admits a reparameterisation in terms of a uniform variable $U\sim\mathcal{U}(0,1)$. We need to first sample a uniform variable in the open interval $(0,1)$ and transform the result to a Kumaraswamy variable via the inverse cdf (10a), then shift and scale the result to cover the stretched support (10b), and finally, apply the rectifier in order to get a sample in the closed interval $[0,1]$ (10c).

[TABLE]

We denote this $h=s(u;a,b,l,r)$ for short. Note that this transformation has two discontinuity points, namely, $t=0$ and $t=1$. Though recall, the probability of sampling $t$ exactly [math] or exactly $1$ is zero, which essentially means stochasticity circumvents points of non-differentiability of the rectifier (see Appendix A.3).

4 Controlled Sparsity

Following Louizos et al. (2017), we relax non-differentiable penalties by computing them on expectation under our latent model $p(z|x,\phi)$. In addition, we propose the use of Lagrangian relaxation to target specific values for the penalties. Thanks to the tractable Kumaraswamy cdf, the expected value of $L_{0}(z)$ is known in closed form

[TABLE]

where $\mathbb{P}(Z_{i}=0)=F_{K}\left(\frac{-l}{r-l};a_{i},b_{i}\right)$. This quantity is a tractable and differentiable function of the parameters $\phi$ of the latent model. We can also compute a relaxation of fused lasso by computing the expected number of zero-to-nonzero and nonzero-to-zero changes:

[TABLE]

In both cases, we make the assumption that latent variables are independent given $x$, in Appendix B.1.2 we discuss how to estimate the regularizers for a model $p(z_{i}|x,z_{<i})$ that conditions on the prefix $z_{<i}$ of sampled $\operatorname{HardKuma}$ assignments.

We can use regularizers to promote sparsity, but just how much text will our final model select? Ideally, we would target specific values $r$ and solve a constrained optimization problem. In practice, constrained optimisation is very challenging, thus we employ Lagrangian relaxation instead:

[TABLE]

where $R(\phi)$ is a vector of regularisers, e.g. expected $L_{0}$ and expected fused lasso, and $\lambda$ is a vector of Lagrangian multipliers $\lambda$. Note how this differs from the treatment of Lei et al. (2016) shown in (4) where regularizers are computed for assignments, rather than on expectation, and where $\lambda_{0},\lambda_{1}$ are fixed hyperparameters.

5 Sentiment Classification

As a concrete example, consider the case of sentiment classification where $x$ is a sentence and $y$ is a $5$-way sentiment class (from very negative to very positive). The model consists of

[TABLE]

where the shape parameters $a,b=g(x;\phi)$, i.e. two sequences of $n$ strictly positive scalars, are predicted by a NN, and the support boundaries $(l,r)$ are fixed hyperparameters.

We first specify an architecture that parameterizes latent selectors and then use a reparameterized sample to restrict which parts of the input contribute encodings for classification:444We describe architectures using blocks denoted by $\operatorname{layer}(\text{inputs};\text{subset of parameters})$, boldface letters for vectors, and the shorthand $\mathbf{v}_{1}^{n}$ for a sequence $\langle\mathbf{v}_{1},\ldots,\mathbf{v}_{n}\rangle$.

[TABLE]

where $\operatorname{emb}(\cdot)$ is an embedding layer, $\operatorname{birnn}(\cdot;\phi_{r})$ is a bidirectional encoder, $f_{a}(\cdot;\phi_{a})$ and $f_{b}(\cdot;\phi_{b})$ are feed-forward transformations with $\operatorname{softplus}$ outputs, and $s(\cdot)$ turns the uniform sample $u_{i}$ into the latent selector $z_{i}$ (see §3). We then use the sampled $z$ to modulate inputs to the classifier:

[TABLE]

where $\operatorname{rnn}(\cdot;\theta_{\text{fwd}})$ and $\operatorname{rnn}(\cdot;\theta_{\text{bwd}})$ are recurrent cells such as LSTMs (Hochreiter and Schmidhuber, 1997) that process the sequence in different directions, and $f_{o}(\cdot;\theta_{o})$ is a feed-forward transformation with $\operatorname{softmax}$ output. Note how $z_{i}$ modulates features $\mathbf{e}_{i}$ of the input $x_{i}$ that are available to the recurrent composition function.

We then obtain gradient estimates of $\mathcal{E}(\phi,\theta)$ via Monte Carlo (MC) sampling from

[TABLE]

where $z=s_{\phi}(u,x)$ is a shorthand for elementwise application of the transformation from uniform samples to HardKuma samples. This reparameterisation is the key to gradient estimation through stochastic computation graphs (Kingma and Welling, 2014; Rezende et al., 2014).

Deterministic predictions.

At test time we make predictions based on what is the most likely assignment for each $z_{i}$. We $\operatorname*{\arg\,\max}$ across configurations of the distribution, namely, $z_{i}=0$, $z_{i}=1$, or $0<z_{i}<1$. When the continuous interval is more likely, we take the expected value of the underlying Kumaraswamy variable.

6 Experiments

We perform experiments on multi-aspect sentiment analysis to compare with previous work, as well as experiments on sentiment classification and natural language inference. All models were implemented in PyTorch, and Appendix B provides implementation details.

Goal.

When rationalizing predictions, our goal is to perform as well as systems using the full input text, while using only a subset of the input text, leaving unnecessary words out for interpretability.

6.1 Multi-aspect Sentiment Analysis

In our first experiment we compare directly with previous work on rationalizing predictions (Lei et al., 2016). We replicate their setting.

Data.

A pre-processed subset of the BeerAdvocate555https://www.beeradvocate.com/ data set is used (McAuley et al., 2012). It consists of 220,000 beer reviews, where multiple aspects (e.g. look, smell, palate) are rated. As shown in Figure 1, a review typically consists of multiple sentences, and contains a 0-5 star rating (e.g. 3.5 stars) for each aspect. Lei et al. mapped the ratings to scalars in $[0,1]$.

Model.

We use the models described in §5 with two small modifications: 1) since this is a regression task, we use a sigmoid activation in the output layer of the classifier rather than a softmax,666From a likelihood learning point of view, we would have assumed a Logit-Normal likelihood, however, to stay closer to Lei et al. (2016), we employ mean squared error. and 2) we use an extra RNN to condition $z_{i}$ on $z_{<i}$:

[TABLE]

For a fair comparison we follow Lei et al. by using RCNN777An RCNN cell can replace any LSTM cell and works well on text classification problems. See appendix B. cells rather than LSTM cells for encoding sentences on this task. Since this cell is not widely used, we verified its performance in Table 1. We observe that the BiRCNN performs on par with the BiLSTM (while using 50% fewer parameters), and similarly to previous results.

Evaluation.

A test set with sentence-level rationale annotations is available. The precision of a rationale is defined as the percentage of words with $z\neq 0$ that is part of the annotation. We also evaluate the predictions made from the rationale using mean squared error (MSE).

Baselines.

For our baseline we reimplemented the approach of Lei et al. (2016) which we call Bernoulli after the distribution they use to sample $z$ from. We also report their attention baseline, in which an attention score is computed for each word, after which it is simply thresholded to select the top-k percent as the rationale.

Results.

Table 2 shows the precision and the percentage of selected words for the first three aspects. The models here have been selected based on validation MSE and were tuned to select a similar percentage of words (‘selected’). We observe that our Bernoulli reimplementation reaches the precision similar to previous work, doing a little bit worse for the ‘look’ aspect. Our HardKuma managed to get even higher precision, and it extracted exactly the percentage of text that we specified (see §4).888We tried to use Lagrangian relaxation for the Bernoulli model, but this led to instabilities (e.g. all words selected). Figure 3 shows the MSE for all aspects for various percentages of extracted text. We observe that HardKuma does better with a smaller percentage of text selected. The performance becomes more similar as more text is selected.

6.2 Sentiment Classification

We also experiment on the Stanford Sentiment Treebank (SST) (Socher et al., 2013). There are 5 sentiment classes: very negative, negative, neutral, positive, and very positive. Here we use the HardKuma model described in §5, a Bernoulli model trained with REINFORCE, as well as a BiLSTM.

Results.

Figure 4 shows the classification accuracy for various percentages of selected text. We observe that HardKuma outperforms the Bernoulli model at each percentage of selected text. HardKuma reaches full-text baseline performance already around 40% extracted text. At that point, it obtains a test score of 45.84, versus 42.22 for Bernoulli and 47.4$\pm 0.8$ for the full-text baseline.

Analysis.

We wonder what kind of words are dropped when we select smaller amounts of text. For this analysis we exploit the word-level sentiment annotations in SST, which allows us to track the sentiment of words in the rationale. Figure 5 shows that a large portion of dropped words have neutral sentiment, and it seems plausible that exactly those words are not important features for classification. We also see that HardKuma drops (relatively) more neutral words than Bernoulli.

6.3 Natural Language Inference

In Natural language inference (NLI), given a premise sentence $x^{(p)}$ and a hypothesis sentence $x^{(h)}$, the goal is to predict their relation $y$ which can be contradiction, entailment, or neutral. As our dataset we use the Stanford Natural Language Inference (SNLI) corpus (Bowman et al., 2015).

Baseline.

We use the Decomposable Attention model (DA) of Parikh et al. (2016).999Better results e.g. Chen et al. (2017) and data sets for NLI exist, but are not the focus of this paper. DA does not make use of LSTMs, but rather uses attention to find connections between the premise and the hypothesis that are predictive of the relation. Each word in the premise attends to each word in the hypothesis, and vice versa, resulting in a set of comparison vectors which are then aggregated for a final prediction. If there is no link between a word pair, it is not considered for prediction.

Model.

Because the premise and hypothesis interact, it does not make sense to extract a rationale for the premise and hypothesis independently. Instead, we replace the attention between premise and hypothesis with HardKuma attention. Whereas in the baseline a similarity matrix is $\operatorname{softmax}$-normalized across rows (premise to hypothesis) and columns (hypothesis to premise) to produce attention matrices, in our model each cell in the attention matrix is sampled from a HardKuma parameterized by $(a,b)$. To promote sparsity, we use the relaxed $L_{0}$ to specify the desired percentage of non-zero attention cells. The resulting matrix does not need further normalization.

Results.

With a target rate of 10%, the HardKuma model achieved 8.5% non-zero attention. Table 3 shows that, even with so many zeros in the attention matrices, it only does about 1% worse compared to the DA baseline. Figure 6 shows an example of HardKuma attention, with additional examples in Appendix B. We leave further explorations with HardKuma attention for future work.

7 Related Work

This work has connections with work on interpretability, learning from rationales, sparse structures, and rectified distributions. We discuss each of those areas.

Interpretability.

Machine learning research has been focusing more and more on interpretability (Gilpin et al., 2018). However, there are many nuances to interpretability (Lipton, 2016), and amongst them we focus on model transparency.

One strategy is to extract a simpler, interpretable model from a neural network, though this comes at the cost of performance. For example, Thrun (1995) extract if-then rules, while Craven and Shavlik (1996) extract decision trees.

There is also work on making word vectors more interpretable. Faruqui et al. (2015) make word vectors more sparse, and Herbelot and Vecchi (2015) learn to map distributional word vectors to model-theoretic semantic vectors.

Similarly to Lei et al. (2016), Titov and McDonald (2008) extract informative fragments of text by jointly training a classifier and a model predicting a stochastic mask, while relying on Gibbs sampling to do so. Their focus is on using the sentiment labels as a weak supervision signal for opinion summarization rather than on rationalizing classifier predictions.

There are also related approaches that aim to interpret an already-trained model, in contrast to Lei et al. (2016) and our approach where the rationale is jointly modeled. Ribeiro et al. (2016) make any classifier interpretable by approximating it locally with a linear proxy model in an approach called LIME, and Alvarez-Melis and Jaakkola (2017) propose a framework that returns input-output pairs that are causally related.

Learning from rationales.

Our work is different from approaches that aim to improve classification using rationales as an additional input (Zaidan et al., 2007; Zaidan and Eisner, 2008; Zhang et al., 2016). Instead, our rationales are latent and we are interested in uncovering them. We only use annotated rationales for evaluation.

Sparse layers.

Also arguing for enhanced interpretability, Niculae and Blondel (2017) propose a framework for learning sparsely activated attention layers based on smoothing the $\max$ operator. They derive a number of relaxations to $\max$, including $\operatorname{softmax}$ itself, but in particular, they target relaxations such as $\operatorname{sparsemax}$ (Martins and Astudillo, 2016) which, unlike $\operatorname{softmax}$, are sparse (i.e. produce vectors of probability values with components that evaluate to exactly [math]). Their activation functions are themselves solutions to convex optimization problems, to which they provide efficient forward and backward passes. The technique can be seen as a deterministic sparsely activated layer which they use as a drop-in replacement to standard attention mechanisms. In contrast, in this paper we focus on binary outcomes rather than $K$-valued ones. Niculae et al. (2018) extend the framework to structured discrete spaces where they learn sparse parameterizations of discrete latent models. In this context, parameter estimation requires exact marginalization of discrete variables or gradient estimation via REINFORCE. They show that oftentimes distributions are sparse enough to enable exact marginal inference.

Peng et al. (2018) propose SPIGOT, a proxy gradient to the non-differentiable $\operatorname*{\arg\,\max}$ operator. This proxy requires an $\operatorname*{\arg\,\max}$ solver (e.g. Viterbi for structured prediction) and, like the straight-through estimator (Bengio et al., 2013), is a biased estimator. Though, unlike ST it is efficient for structured variables. In contrast, in this work we chose to focus on unbiased estimators.

Rectified Distributions.

The idea of rectified distributions has been around for some time. The rectified Gaussian distribution (Socci et al., 1998), in particular, has found applications to factor analysis (Harva and Kaban, 2005) and approximate inference in graphical models (Winn and Bishop, 2005). Louizos et al. (2017) propose to stretch and rectify samples from the BinaryConcrete (or GumbelSoftmax) distribution (Maddison et al., 2017; Jang et al., 2017). They use rectified variables to induce sparsity in parameter space via a relaxation to $L_{0}$. We adapt their technique to promote sparse activations instead. Rolfe (2017) learns a relaxation of a discrete random variable based on a tractable mixture of a point mass at zero and a continuous reparameterizable density, thus enabling reparameterized sampling from the half-closed interval $[0,\infty)$. In contrast, with HardKuma we focused on giving support to both 0s and 1s.

8 Conclusions

We presented a differentiable approach to extractive rationales, including an objective that allows for specifying how much text is to be extracted. To allow for reparameterized gradient estimates and support for binary outcomes we introduced the HardKuma distribution. Apart from extracting rationales, we showed that HardKuma has further potential uses, which we demonstrated on premise-hypothesis attention in SNLI. We leave further explorations for future work.

Acknowledgments

We thank Luca Falorsi for pointing us to Louizos et al. (2017), which inspired the HardKumaraswamy distribution. This work has received funding from the European Research Council (ERC StG BroadSem 678254), the European Union’s Horizon 2020 research and innovation programme (grant agreement No 825299, GoURMET), and the Dutch National Science Foundation (NWO VIDI 639.022.518, NWO VICI 277-89-002).

Appendix A Kumaraswamy distribution

A Kumaraswamy-distributed variable $K\sim\operatorname{Kuma}(a,b)$ takes on values in the open interval $(0,1)$ and has density

[TABLE]

where $a\in\mathbb{R}_{>0}$ and $b\in\mathbb{R}_{>0}$ are shape parameters. Its cumulative distribution takes a simple closed-form expression

[TABLE]

with inverse

[TABLE]

A.1 Generalised-support Kumaraswamy

We can generalise the support of a Kumaraswamy variable by specifying two constants $l<r$ and transforming a random variable $K\sim\operatorname{Kuma}(a,b)$ to obtain $T\sim\operatorname{Kuma}(a,b,l,r)$ as shown in (20, left).

[TABLE]

The density of the resulting variable is

[TABLE]

where $r-l>0$ by definition. This affine transformation leaves the cdf unchanged, i.e.

[TABLE]

Thus we can obtain samples from this generalised-support Kumaraswamy by sampling from a uniform distribution $\mathcal{U}(0,1)$, applying the inverse transform (19), then shifting and scaling the sample according to (20, left).

A.2 Rectified Kumaraswamy

First, we stretch a Kumaraswamy distribution to include [math] and $1$ in its support, that is, with $l<0$ and $r>1$, we define $T\sim\operatorname{Kuma}(a,b,l,r)$. Then we apply a hard-sigmoid transformation to this variable, that is, $h=\min(0,\max(1,t))$, which results in a rectified distribution which gives support to the closed interval $[0,1]$. We denote this rectified variable by

[TABLE]

whose distribution function is

[TABLE]

where

[TABLE]

is the probability of sampling exactly [math], where

[TABLE]

is the probability of sampling exactly 1, and

[TABLE]

is the probability of drawing a continuous value in $(0,1)$. Note that we used the result in (22) to express these probabilities in terms of the tractable cdf of the original Kumaraswamy variable.

A.3 Reparameterized gradients

Let us consider the case where we need derivatives of a function $\mathcal{L}(u)$ of the underlying uniform variable $u$, as when we compute reparameterized gradients in variational inference. Note that

[TABLE]

by chain rule. The term $\partialderivative{\mathcal{L}}{h}$ depends on a differentiable observation model and poses no challenge; the term $\partialderivative{h}{t}$ is the derivative of the hard-sigmoid function, which is [math] for $t<0$ or $t>1$, $1$ for $0<t<1$, and undefined for $t\in\{0,1\}$; the term $\partialderivative{t}{k}=r-l$ follows directly from (20, left); the term $\partialderivative{k}{u}=\partialderivative{u}F^{-1}_{K}(u;a,b)$ depends on the Kumaraswamy inverse cdf (19) and also poses no challenge. Thus the only two discontinuities happen for $t\in\{0,1\}$, which is a [math] measure set under the stretched Kumaraswamy: we say this reparameterisation is differentiable almost everywhere, a useful property which essentially circumvents the discontinuity points of the rectifier.

A.4 HardKumaraswamy PDF and CDF

Figure 8 plots the pdf of the HardKumaraswamy for various a and b parameters. Figure 9 does the same but with the cdf.

Appendix B Implementation Details

B.1 Multi-aspect Sentiment Analysis

Our hyperparameters are taken from Lei et al. (2016) and listed in Table 4. The pre-trained word embeddings and data sets are available online at http://people.csail.mit.edu/taolei/beer/. We train for 100 epochs and select the best models based on validation loss. For the MSE trade-off experiments on all aspects combined, we train for a maximum of 50 epochs.

For the Bernoulli baselines we vary $L_{0}$ weight $\lambda_{1}$ among $\{0.0002,0.0003,0.0004\}$, just as in the original paper. We set the fused lasso (coherence) weight $\lambda_{2}$ to $2*\lambda_{1}$.

For the HardKuma models we set a target selection rate to the values targeted in Table 2, and optimize to this end using the Lagrange multiplier. We chose the fused lasso weight from $\{0.0001,0.0002,0.0003,0.0004\}$.

B.1.1 Recurrent Unit

In our multi-aspect sentiment analysis experiments we use the RCNN of Lei et al. (2016). Intuitively, the RCNN is supposed to capture n-gram features that are not necessarily consecutive. We use the bigram version (filter width $n=2$) used in Lei et al. (2016), which is defined as:

[TABLE]

B.1.2 Expected values for dependent latent variables

The expected $L_{0}$ is a chain of nested expectations, and we solve each term

[TABLE]

as a function of a sampled prefix, and the shape parameters $a_{i},b_{i}=g_{i}(x,z_{<i};\phi)$ are predicted in sequence.

B.2 Sentiment Classification (SST)

For sentiment classification we make use of the PyTorch bidirectional LSTM module for encoding sentences, for both the rationale extractor and the classifier. The BiLSTM final states are concatenated, after which a linear layer followed by a softmax produces the prediction. Hyperparameters are listed in Table 5. We apply dropout to the embeddings and to the input of the output layer.

B.3 Natural Language Inference (SNLI)

Our hyperparameters are taken from Parikh et al. (2016) and listed in Table 6. Different from Parikh et al. is that we use Adam as the optimizer and a batch size of 64. Word embeddings are projected to 200 dimensions with a trained linear layer. Unknown words are mapped to 100 unknown word classes based on the MD5 hash function, just as in Parikh et al. (2016), and unknown word vectors are randomly initialized. We train for 100 epochs, evaluate every 1000 updates, and select the best model based on validation loss. Figure 10 shows a correct and incorrect example with HardKuma attention for each relation type (entailment, contradiction, neutral).