TACAM: Topic And Context Aware Argument Mining

TL;DR

This paper introduces TACAM, a novel argument mining approach that incorporates topic and external context information, significantly improving the accuracy of identifying arguments within large text corpora.

Contribution

It proposes models that integrate topic context and external knowledge sources for argument classification, advancing argument mining beyond traditional topic-independent methods.

Findings

Topic-aware models outperform baseline approaches.

External knowledge integration boosts argument classification accuracy.

Contextual models significantly improve argument detection performance.

Abstract

In this work we address the problem of argument search. The purpose of argument search is the distillation of pro and contra arguments for requested topics from large text corpora. In previous works, the usual approach is to use a standard search engine to extract text parts which are relevant to the given topic and subsequently use an argument recognition algorithm to select arguments from them. The main challenge in the argument recognition task, which is also known as argument mining, is that often sentences containing arguments are structurally similar to purely informative sentences without any stance about the topic. In fact, they only differ semantically. Most approaches use topic or search term information only for the first search step and therefore assume that arguments can be classified independently of a topic. We argue that topic information is crucial for argument mining, since the topic defines the semantic context of an argument. Precisely, we propose different models for the classification of arguments, which take information about a topic of an argument into account. Moreover, to enrich the context of a topic and to let models understand the context of the potential argument better, we integrate information from different external sources such as Knowledge Graphs or pre-trained NLP models. Our evaluation shows that considering topic information, especially in connection with external information, provides a significant performance boost for the argument mining task.

Tables

Table 1. Table 1. In-Topic

	Method
two-class	BiLSTM	0.74
	BiCLSTM	0.74
	TACAM-WE	0.74
	TACAM-KG	0.73
	CAM-BERT Base	0.79
	TACAM-BERT Base	0.81
	CAM-BERT Large	0.80
	TACAM-BERT Large	0.81
three-class	BiLSTM	0.56
	BiCLSTM	0.53
	TACAM-WE	0.54
	TACAM-KG	0.56
	CAM-BERT Base	0.65
	TACAM-BERT Base	0.66
	CAM-BERT Large	0.67
	TACAM-BERT Large	0.69

Table 2. Table 2. Cross-Topic

	Method	Topics
		Abortion	Cloning	Death penalty	Gun control	Marij. legal.	Min. wage	Nucl. energy	School unif.	$⌀$
two-classes	BiLSTM	0.61	0.72	0.70	0.75	0.64	0.62	0.67	0.54	0.66
	BiCLSTM	0.67	0.71	0.71	0.73	0.69	0.75	0.71	0.58	0.70
	TACAM-WE	0.64	0.71	0.70	0.74	0.64	0.63	0.68	0.55	0.66
	TACAM-KG	0.62	0.69	0.70	0.75	0.64	0.76	0.71	0.56	0.68
	CAM-BERT Base	0.61	0.77	0.74	0.76	0.74	0.61	0.76	0.73	0.72
	CAM-BERT Large	0.62	0.79	0.75	0.77	0.77	0.65	0.75	0.73	0.73
	TACAM-BERT Base	0.78	0.77	0.78	0.80	0.79	0.83	0.80	0.83	0.80
	TACAM-BERT Large	0.79	0.78	0.78	0.81	0.79	0.84	0.83	0.82	0.80
three-classes	BiLSTM	0.47	0.52	0.48	0.48	0.44	0.42	0.48	0.42	0.46
	BiCLSTM	0.49	0.52	0.46	0.51	0.46	0.44	0.47	0.42	0.47
	TACAM-WE	0.47	0.52	0.47	0.48	0.46	0.46	0.48	0.41	0,47
	TACAM-KG	0.46	0.51	0.47	0.47	0.46	0.48	0.47	0.41	0.47
	CAM-BERT Base	0.38	0.63	0.53	0.49	0.54	0.54	0.61	0,50	0.53
	TACAM-BERT Base	0.42	0.68	0.54	0.50	0.60	0.49	0.64	0.69	0.57
	CAM-BERT Large	0.53	0.67	0.56	0.53	0.59	0.66	0.67	0.66	0.61
	TACAM-BERT Large	0.54	0.69	0.59	0.55	0.63	0.69	0.71	0.69	0.64

Table 3. Table 3. Related terms for each topic

Topic	Related terms
abortion	euthanasia	teenage pregnancy	family	medical procedure	rape
cloning	biology	species	religion	organ donation	modified food
death penalty	politics	ethic	prison	homicide	sentence
gun control	safety	school shooting	robbery	regulation	police state
marijuana legalization	drugs	medicine	relaxation	freedom	liberty
minimum wage	social justice	slavery	automation	economic crisis	stagnation
nuclear energy	environment	employment	industry	pollution	climate change
school uniforms	equality	social justice	individualism	clothing	mobbing

Table 4. Table 4. Topic dependent cross-topic classification results

	Method	Topics
		Abortion	Cloning	Death penalty	Gun control	Marij. legal.	Min. wage	Nucl. energy	School unif.	$⌀$
two-classes	BiLSTM	0.57	0.59	0.53	0.59	0.62	0.62	0.59	0.57	0.58
	BiCLSTM	0.62	0.72	0.46	0.46	0.76	0.60	0.69	0.45	0.60
	CAM-BERT Base	0.56	0.63	0.60	0.62	0.61	0.55	0.60	0.53	0.59
	TACAM-BERT Base	0.68	0.77	0.78	0.79	0.82	0.85	0.79	0.58	0.76
three-classes	BiLSTM	0.39	0.39	0.37	0.36	0.39	0.42	0.40	0.39	0.39
	BiCLSTM	0.46	0.34	0.29	0.35	0.42	0.29	0.47	0.30	0.36
	CAM-BERT Base	0.42	0.50	0.42	0.42	0.48	0.51	0.50	0.49	0.47
	TACAM-BERT Base	0.44	0.60	0.52	0.49	0.61	0.65	0.62	0.55	0.56

Equations

$$x^s$$

$$h^s$$

$$x^t$$

$$h^t$$

$$h_l$$

$$\hat{y}$$

Full text

transfer learning, argument mining, argument search, natural language processing

††copyright: acmcopyright††journalyear: 2019††conference: IEEE/WIC/ACM International Conference on Web Intelligence; October 14–17, 2019; Thessaloniki, Greece††booktitle: IEEE/WIC/ACM International Conference on Web Intelligence (WI ’19), October 14–17, 2019, Thessaloniki, Greece††price: 15.00††doi: 10.1145/3350546.3352506††isbn: 978-1-4503-6934-3/19/10

1. Introduction

The main focus of argument search lies on presenting an overview of different standpoints and their justifications to some inquired topic e.g. cloning or minimum wages. This may be useful in different scenarios, like legal reasoning (Wyner et al., 2010) or decision making processes (Svenson, 1979), especially if a topic or a problem is controversial. An automated argument search process could ease much of the manual effort involved in these areas, especially if it can make use of large text databases or even combinations of them. The online argument search in state-of-the-art argument search systems proceeds in two steps (Stab et al., 2018a):

1. (1)

Some standard text search engine, e.g. (Białecki et al., 2012), extracts relevant text parts from large text corpora using a given topic as a query. 2. (2)

Relevant text parts are analyzed sentence-wise by an argument recognition component which decides for each sentence whether it is an argument and optional about its stance.

Therefore, the core technique in argument search is argument recognition or argument mining (Peldszus and Stede, 2013; Lippi and Torroni, 2015b; Swanson et al., 2015; Villalba and Saint-Dizier, 2012; Lippi and Torroni, 2015a). The basis for argument mining is an argument model (to avoid confusion with machine learning models in the following we refer to argument model as argument scheme). An argument scheme formally defines what kind of arguments exist and what their properties and relationships between them are. State-of-the-art argument search systems work with simple argument models without relationships between arguments. Therefore, the common task of a machine learning model is argument recognition or identification.

The classic argument recognition approaches extract arguments from text without taking the topic of the argument into consideration (Palau and Moens, 2009; Habernal and Gurevych, 2016; Eger et al., 2017). However, the special characteristic of application of argument recognition in argument search is that there is always a query topic. The query topic carries information about the query context and understanding the context of potential arguments can be crucial for the decision. For instance, if the query is about the usefulness of some medical procedure in the context of medicine, we expect appropriate arguments from the medical doctors and not from people who share their own individual experiences. Thus, if a potential argument follows a particular structure or some special terminology is used, it may increase an argument’s chances to be classified as an argument.

Another desirable property of an argument identification approach is to be able to decide dependent on a topic, whether a sentence is an argument. To be useful, an argument search engine heavily relies on a large text corpus. The larger the text corpus is, the more probable is the scenario that texts extracted by the search engine contain arguments about different topics, which increases coverage of different topics but further complicates the argument classification. For instance, consider the following example: A user looks for arguments to Emission Trading System (ETS) and the following sentence candidates are retrieved by the text search engine:

• •

ETS sets a clear price on carbon and combats climate change.

• •

Free trade secures all the advantages of international division of labour.

• •

UK signals plan to leave EU emissions trading scheme after Brexit.

The first two sentences can be considered as arguments, the third sentence is purely informative and it does not persuade towards any stance. However, if we look more closely at the second sentence, we recognize that this is not an argument about the query topic Emission Trading. This is obvious for humans, since we understand that the context of Free Trade is different from the Emission Trading context, even if both contexts are related to Trade. Therefore, the better the machine learning model is able to grasp the context of a topic and of potential arguments at different granularities, the better is the decision the model can make and the more certain it can be about its decisions. Considering relationship between potential argument and topic is different from the classical relation detection task in argument mining. The input to relation detection algorithm are parts of the text, which are already recognized as arguments and presence or absence of some relationships does not affect decision about text parts of being argumentative. In contrast, our approach takes the relationship to the topic into account when deciding whether a text is an argument.

In this work we propose a new approach for argument mining which also takes the topic of potential arguments into account. The overview of our approach is depicted in Figure 1. The standard argument search pipeline looks like the workflow presented in the figure without the context source and the dotted arrows. Our approach enriches the argument candidates with the context and topic information in the classification process. We show how the contextual information about a topic and an argument from different sources like knowledge graphs or pre-trained models can be integrated into our approach. We investigate the benefits of considering the topic and the integration of external knowledge. We summarize our main contributions as follows:

• •

We present a novel approach for argument classification which takes the topic of the argument into account by extending the methodology introduced by the authors of (Stab et al., 2018c).

• •

We show how contextual information about topic and argument from different sources like knowledge graphs or pre-trained models can be integrated.

• •

We demonstrate that considering topics is beneficial for the argument classification, especially in connection with external knowledge.

• •

We show that our approach is particularly successful if the model has to generalize to unseen topics. Since we cannot expect that available training datasets for argument recognition cover all possible topics, the generalization to unseen topics is an important requirement.

• •

We present thorough experimental evaluations of our models and comparisons to state-of-the-art methods on a real-world dataset and introduce an additional experimental setting. In this setting we evaluate the ability of different models to classify in the context of topics.

2. Related Work

In general, the main focus in argument mining lies in the recognition of argument components (Palau and Moens, 2009; Habernal and Gurevych, 2016; Stab and Gurevych, 2017; Hua and Wang, 2017; Nguyen and Litman, 2015) and the detection of relations between them (Stab and Gurevych, 2017; Nguyen and Litman, 2016). However, all these approaches which tackle the problem of argument classification do not take information about the specific topic of a given argument into consideration.

At the same time different argumentation schemes of different complexity were proposed in previous works (Toulmin, 1958; Walton, 2012; Freeman, 2011; Stab and Gurevych, 2014). Since each argumentation scheme contains different numbers of various argument types, this has an implication on machine learning models designed for argument detection, since they have to learn how to identify them. However, as was shown in (Daxenberger et al., 2017), these argumentation schemes do not generalize well to different types of texts. Concretely, the authors of this work collected datasets used with different argumentation schemes and combined them in a single dataset. Afterwards, they trained a model, which should detect the argument component of type claim, which is central in each argument scheme. However, the machine learning models which perform well for single datasets could not achieve good results on this simple binary classification task. Additionally, it was shown that even human annotators often label differently when annotating the same datasets according to complex argumentation schemes. Therefore, the authors came to the conclusion that certain argument components (backing, warrant) as introduced in (Toulmin, 1958), and other argumentation schemes are often only stated implicitly in common argumentation documents on the internet. In more recent work, argumentation schemes became simpler and more flexible (Stab et al., 2018c; Wachsmuth et al., 2017). This enables broader applicability and topic-dependent argument search across multiple text types.

There are various approaches to consider context in argument mining. Hand-crafted features extracted from source text were used for argument classification (Nguyen and Litman, 2015) and relation detection (Nguyen and Litman, 2016). More related to our work is a method presented in (Stab et al., 2018c). The authors introduced a dataset with arguments of different text types and topics for each argument. Additionally, they propose two simple argumentation schemes. The first scheme is a binary decision, aiming at classifying a sentence as argumentative or non-argumentative. In the second scheme there is a distinction between non argumentative sentences and pro and contra arguments. They also propose a model which takes topics into consideration. We extend their work by proposing new architectures and context sources and compare our approach with their method.

There are few approaches which use transfer learning for the argument mining task. In (Stab et al., 2018c) the proposed model is pre-trained on another dataset for argument mining (Habernal et al., 2016), but this approach does not lead to considerable improvement. Parallel to our work, the authors of (Trautmann et al., 2019) also use transfer learning with BERT for a new introduced corpus with tagged sequences. However, their model does not generalize to the new topics by design.

Based on recent developments two argument search engines, i.e., www.args.me (Wachsmuth et al., 2017) and www.argumentsearch.com (Stab et al., 2018b), where a user is able to search a broad range of documents for certain topics, have been developed.

3. Problem setting

We model the recognition of argumentative sentences as a classification task. Given a sentence $s=\{s_{0},\dots,s_{n}\}$ and a topic $t=\{t_{0},\dots,t_{k}\}$ with $s_{i}\in\{0,1\}^{V}$, $t_{i}\in\{0,1\}^{V}$ being one-hot encoded vectors, and $V$ being the size of the vocabulary, we seek to classify $s$ as ”contra argument” or ”pro argument” if the sentence $s$ includes evidence for supporting or opposing the topic $t$. If the sentence does not contain evidence, it is classified as a ”non-argument”.

4. Method

In contrast to previous approaches, we aim at incorporating context information into the learning procedure when training our models. This way, the models learn which argument properties are especially meaningful in the context of a particular topic and can put a special emphasis on these information for the subsequent classification task. For instance, emission trading is a frequently discussed topic, but we would expect the most meaningful arguments about its usefulness coming from particular academic communities. Consequently, by providing topic information in a meaningful way, we enable models e.g. to learn argument structures and vocabulary which are common in those communities. On the other hand we also expect our models to learn how topics are related to their domain specific arguments. Although a sentence might contain topic-specific words it may still be an argument of a different topic. Considering the topic emission trading again, relevant arguments are probably more related to climate change than to the stock market, though trading is a frequently used term in the latter area. Thus, it is important to understand the context of the topic and the context of the potential arguments. Consequently, we propose various approaches to provide context information about topic and potential argument from various external sources. However, as the proposed models should be able to generalize to arbitrary topics, we provide the context information as an additional input to the models. Therefore, all our models aggregate the representation of the potential argument with the representation of the topic.

4.1. Models

4.1.1. Recurrent Network

The first model we propose is a recurrent model for which we use two instances of a BiLSTM (Hochreiter and Schmidhuber, 1997) model. Precisely, one is used to encode a topic and the other model aims at encoding the potential argument:

[TABLE]

We use word2vec (Mikolov et al., 2013) embeddings $W^{we}\in\mathbb{R}^{V\times d}$ of the given words in a sentence $s$ as input for the argument BiLSTM instance $BiLSTM_{a}$. However, it is noteworthy that any other kind of word embeddings can be used, too. Furthermore, function $f_{map}$ maps some given topic description $t$ to a sequence of entities $x^{t}$. In general, we allow arbitrary information sources to provide topic context. Therefore, $f_{map}$ depends on the information source. In case of describing the relevant entities of $t$ in terms of relevant words, one could use a sequence of word embeddings to encode the topic information. In this case $f_{map}$ would map the relevant words to the corresponding one-hot encoded vectors which, if multiplied with the word embedding matrix $W^{we}$, serve as input for the topic BiLSTM instance denoted as $BiLSTM_{t}$. In case of using knowledge graphs as external source of information for the context, $f_{map}$ first examines whether there is an entity with the same name as the whole topic description. Otherwise it maps each word in the topic description to an corresponding entity in the knowledge graph. If there is no such corresponding entity for a particular word, we employ a nearest neighbor search for this word in the word embedding space and finally use a knowledge graph entity which matches to a semantically similar word. Once we found an entity for each word in the topic description, we use the corresponding sequence of knowledge graph entity representations as input for the topic BiLSTM instance. The function $aggr$ is used to aggregate topic and argument representations. We evaluate the following aggregation functions:

• •

Addition: $aggr(h^{s},h^{t})=h^{s}+h^{t}$

• •

Hadamard product: $aggr(h^{s},h^{t})=h^{s}\odot h^{t}$

• •

Concatenation: $aggr(h^{s},h^{t})=concat(h^{s},h^{t})$

Finally, we use the aggregated representation $h_{l}$ as input to a dense layer with softmax activation to obtain the classification result $\hat{y}$.

4.1.2. Attention model

We also use a deep bidirectional transformer encoder (Vaswani et al., 2017), the architecture which was used in BERT (Devlin et al., 2018). Specifically, we concatenate argument and topic description and use a special separator token and segment embeddings to distinguish between topic and potential argument. The output of the first special [CLS] token is used as input to the dense classification layer, which predicts the distribution over the classes.

4.2. Context source

As mentioned previously, our models are able to rely on different external sources that may provide the context information. In this work, we experiment with the following sources:

• •

Shallow Word Embeddings (Mikolov et al., 2013; Pennington et al., 2014; Bojanowski et al., 2017) are widely used in NLP applications and encode context information implicitly. In fact, the word embeddings are learned such that the representations of words that frequently appear in similar contexts are similar to each other. We use shallow word embeddings trained by word2vec as input to the recurrent model.

• •

Knowledge Graphs model information about the world explicitly in the form of an heterogeneous graph. The entities in the knowledge graph are represented as nodes, and relationships between them as edges of different types. Information in a knowledge graph is represented as triples consisting of subject, predicate and object, where subject and object are entities and predicate stands for the relationship between them. In contrast to information contained in text data, knowledge graphs are structured, i.e., each entity and relationship have a distinct meaning, and the information about the modelled world are distilled in form of facts. These facts can be extracted from texts, different databases or inserted manually. The trustworthiness of these facts in publicly available knowledge graphs is in general very high (Nickel et al., 2015). In our work we use the english version of the DBpedia knowledge graph, which has about 400 million facts with more than 3.7 million unique entities (Lehmann et al., 2015). We applied TransE (Bordes et al., 2013) to obtain embeddings for the knowledge graph entities. These embeddings are used as input to a recurrent model (alternatively to the word embeddings).

• •

Fine-Tuning based Transfer Learning approaches (Radford et al., 2018; Devlin et al., 2018; Radford et al., 2019) adapt whole models, that were pre-trained on some (auxiliary) task, to a new problem. This is different from feature-based approaches which provide pre-trained representations (Peters et al., 2018; Dai and Le, 2015) and require task-specific architecture for a new problem. We use the weights of pre-trained BERT (Large and Base) (Devlin et al., 2018) models for initializing our 4.1.2 model and train it for the argument classification task.

5. Evaluation

5.1. Dataset and Evaluation Tasks

For the evaluation we use the UKP Sentential Argument Mining corpus from (Stab et al., 2018c). The dataset consists of more than 25000 sentences from multiple text types covering eight different topics. It contains a broad range of genres including news reports, editorials, blogs, debate forums and encyclopedia articles which are all related to at least one topic. The topics have been randomly selected from a list111https://www.questia.com/library/controversial-topics of controversial topics. The authors define an argument as a sentence that can be used to oppose or support a given topic. For all models each sentence is truncated to 60 words according to the experiment setting in (Stab et al., 2018c). Note that in contrast to (Stab et al., 2018c) we use weighted cross-entropy to account for class imbalance.222We assume this is a reason we obtained better results for the comparison methods as stated in the original paper. Following (Stab et al., 2018c) we evaluate our approach by performing the following classification tasks:

• •

Binary classification: whether a sentence is an argument for the given topic.

• •

Multiclass classification: whether a sentence is supporting, respectively attacking an argument, or is not an argument at all for the given topic.

As suggested in (Stab et al., 2018c), we evaluate all approaches in two different scenarios. In the In-Topic scenario each topic is split into training and test data, which leads to arguments of the same topics in both training and test data. The Cross-Topic scenario primarily aims at evaluating the generalization of the models, i.e., answering the question how good the performance of the models is on yet unseen topics. Therefore, seven topics are used for training and the remaining one for test. Let us mention that although Cross-Topic is the more complex task, it is more relevant for real-world problems: The reason is that in general we cannot expect all possible topic queries to be present in a dataset that is available for training.

5.2. Models

For all tasks we compare the following approaches:

• •

BiLSTM is a bidirectional LSTM model (Hochreiter and Schmidhuber, 1997), which does not use topic information

• •

BiCLSTM is the contextual biderectional LSTM (Ghosh et al., 2016). Topic information is used as an additional input to the gates of an LSTM cell. We use the version from (Stab et al., 2018c) where the topic information is only used at the $i-$ and $c-$gates since this model showed the most promising result in their work.

• •

TACAM-WE is our recurrent model described in 4.1.1 which uses word embeddings to define the context of the topic

• •

TACAM-KG is our recurrent model described in 4.1.1 which uses Knowledge Graphs embeddings from DBPedia to define the context of the topic.

• •

TACAM-BERT Base $/$ TACAM-BERT Large are our attention based models with topic information described in Section 4.1.2. Both model use pre-initialized weights (cf. Section 4.2). TACAM-BERT Base has 1/3 parameters of TACAM-BERT Large .

• •

CAM-BERT Base$/$ CAM-BERT Large are similar to TACAM-BERT Base and TACAM-BERT Large models without topic information. These models enrich only potential argument with the context, but do not have access to the topic. Comparing them with their counterparts with topic information enables the evaluation of topic importance.

In our experimental setting we mostly follow the experimental settings suggested in (Stab et al., 2018c). We use the same train/validation/test splits. The validation set is used to select the hyperparameters and we report Macro F1 scores on test sets. To avoid effects of bad initialization and local minima we train each model 10 times and select the model which performs best on the validation set.

5.3. In-topic Results

The results of the in-topic argument classification are listed in Table 1. In this setting we do not expect a large improvement by providing topic information since the models have already been trained with arguments of the same topics as in the training set. The results in Table 1 reflect our expectations: we can slightly improve the classification results for the more complex multiclass classification problem. However, we see a relative increase of about 10% for the two-classes and 20% for the three-classes classification problem by using context information from transfer learning. Therefore, we conclude that contextual information about potential arguments is important and since the topics are diverse, the model is able to learn argument structure for each topic.

5.4. Cross-Topic Results

Our cross-topic results are presented in Table 2. In this experiment, which reflects a real-life argument search scenario, we want to prove our two hypotheses:

• •

When classifying potential arguments, it is advantageous to take information about the topic into account.

• •

The context of an argument and topic context are important for the classification decision.

On the whole, we can see that our two hypotheses are confirmed. In the two-classes scenario the recurrent model improves if topic information is provided by knowledge graph embeddings. By using attention-based models with pre-trained weights we can observe a significant performance boost of eleven score points in average when considering topic information. However, the same model without topic information performs only slightly better than the recurrent models. Therefore, we conclude that both, topic information together with contexts of topic and argument, are important for the correct decision about a potential argument. We observe similar effects in the three-classes scenario. Although in average different contexts for the recurrent model have a similar effect, we can clearly observe that taking topic information into account improves classification results by one score points. The combination of transfer learning for context and topic information again outperforms all other approaches by far. At the same time, the pre-trained model without topic information achieves a macro-f1 score of 0.61 which is 3 points lower than with topic information.

5.5. Topic Dependent Cross-Topic Results

As was shown in the previous subsection, argument classification produces satisfying results, especially if topic information and contexts are taken into account. In this set of experiments we evaluate the ability of different models to classify dependent on the topic. Therefore, a sentence may be considered to be an argument for one topic but be non argumentative for another. We argue that this is important, especially if text corpora are large, to filter out argumentative candidates which are arguments for different topics. To evaluate the models ability to perform well in topic dependent classification we extend our dataset and change the experimental setting. For each topic we select a number of related terms. These are words which come from a similar context as a topic but it is very unlikely that the topic’s argument are valid arguments for them. The list of related terms for each topic is provided in Table 3. For 50% of argumentative sentences selected randomly from the test set, we replace the topic by one of the related terms of the topic and change the sentence label in the test set to non-argumentative. Therefore, to perform well on this task, a model should be able to recognize argumentative sentences in the context of the topic. To train for this task we correspondingly augment the training data. We keep the original training data and additionally select 50% of argumentative sentences from the training set, select one of the related terms as topic, label them as non-argumentative and insert them into the training set. For this task we compare our model, which performed best on the original cross-topic task and compare it with the state-of-the-art approach BiCLSTM . We also include the same models without topic information to see, whether topic information is still helpful or if the models get confused instead.

The results for topic dependent classification are presented in Table 4. For the two-classes problem we observe a massive performance drop of ten points in macro-f1 score for the BiCLSTM model. Nonetheless, the model still makes use of topic information and outperforms the standard BiLSTM by two macro-f1 score points. Our approach TACAM-BERT Base is more robust, the performance falls by moderate four score points and the gap to the counterpart model without topic information is incredible 17 score points large. We observe a similar behaviour in the three-classes scenario. Our TACAM-BERT Base approach achieves the same average score as in the original cross topic task. In contrast the performance of the BiCLSTM model drops by 11 score points and it even performs worse than the same model without topic information on this more complex task. Thus we conclude that unlike previous models our approaches are indeed able to grasp the context of the argument and topic and are able to relate them with each other.

6. Conclusion

In this paper, we introduce a new approach for argument mining which takes a topic of the potential argument into account. We hypothesize that considering information about the topic of a potential argument and their contexts should lead to better argument recognition. We present multiple ways to include topic and contexts into the argument mining process. Precisely, we show how contexts from word embeddings, Knowledge Graph embeddings and models pre-trained on other tasks can be integrated into our approach. Our experimental results clearly show that considering topics in the decision process leads to better results in almost all considered cases. Especially our approach with topic information in connection with context from pre-trained models improves state-of-the-art approach by far in the real-world scenario. We also show that in contrast to current state-of-the-art methods, our approach is robust and able to perfectly grasp the context of topic and potential argument. For future work we plan to focus more on Knowledge Graphs and other external context sources. In detail, we want to use information gathered from knowledge graphs not only for topics but also on the argument side. We also plan to investigate different Knowledge Graph embedding techniques and combine different Knowledge Graphs in the same model. For instance, a combination of fact based knowledge graphs like DBPedia (Lehmann et al., 2015) and Wikidata (Vrandečić and Krötzsch, 2014) with knowledge graphs like WordNet (Miller, 1995) and FrameNet (Baker et al., 1998a, b) which focus on lexical similarities could further increase the representation quality of the context. Additional datasets with topic information about more topics will also deepen our understanding of the interplay between context and arguments and potentially further increase the performance of the argumentation models.

Acknowledgements.

This work has been funded by the Deutsche Forschungsgemeinschaft (DFG) within the project Relational Machine Learning for Argument Validation (ReMLAV), Grant Number SE 1039/10-1, as part of the Priority Program ”Robust Argumentation Machines (RATIO)” (SPP-1999). This work has also been funded by the German Federal Ministry of Education and Research (BMBF) under Grant No. 01IS18036A. The authors of this work take full responsibilities for its content.