Derivational Morphological Relations in Word Embeddings

Tom\'a\v{s} Musil; Jon\'a\v{s} Vidra; David Mare\v{c}ek

arXiv:1906.02510·cs.CL·June 7, 2019

Derivational Morphological Relations in Word Embeddings

Tom\'a\v{s} Musil, Jon\'a\v{s} Vidra, David Mare\v{c}ek

PDF

TL;DR

This paper investigates how word embeddings can reveal derivational relationships in Czech, using clustering of embedding differences to match semantic categories of word derivations.

Contribution

It demonstrates that unsupervised clustering of embedding differences effectively captures derivational relations and semantic categories in morphologically rich Czech.

Findings

01

Clusters largely match manually annotated semantic categories

02

Embedding differences can identify derivational relations

03

Method works well in morphologically rich language

Abstract

Derivation is a type of a word-formation process which creates new words from existing ones by adding, changing or deleting affixes. In this paper, we explore the potential of word embeddings to identify properties of word derivations in the morphologically rich Czech language. We extract derivational relations between pairs of words from DeriNet, a Czech lexical network, which organizes almost one million Czech lemmata into derivational trees. For each such pair, we compute the difference of the embeddings of the two words, and perform unsupervised clustering of the resulting vectors. Our results show that these clusters largely match manually annotated semantic categories of the derivational relations (e.g. the relation 'bake--baker' belongs to category 'actor', and a correct clustering puts it into the same cluster as 'govern--governor').

Tables5

Table 1. Table 1: Classes of Czech derivations.

POS	class	syntactic change
A $\to$ D	adjective $\to$ adverb	-ý +y, -í +ě, -ý +ě, -ý +e
A $\to$ N	designation	-ý +ec, -ý +ka
A $\to$ N	feature	-í +ost, -ý +ost
A $\to$ N	subject	-ký +tví
N $\to$ A	pertaining to	+ový, -a +ový, +ní, -a +ní, -ce +ční, +ný, +ský, -e +cký, -ka +cký
N $\to$ A	possessive	+ův, -a +in, -o +ův, -ek +kův, -a +ův
N $\to$ N	diminutization	+ek, -k +ček
N $\to$ N	instrument / scientist	-ie
N $\to$ N	man $\to$ woman	-a +ová, +ka, +ová, +vá, -ý +á, -ík +ice
N $\to$ N	man $\to$ woman / diminutization	-a +ka
N $\to$ N/A $\to$ A	super	super+
N $\to$ V	noun $\to$ verb	+ovat
V $\to$ A	ability	+elný
V $\to$ A	acting	-it +ící, -ovat +ující, -t +jící
V $\to$ A	general property	-t +vý
V $\to$ A	patient	-t +ný, -it +ený, -it +ěný, -nout +lý, -t +lý, -out +utý
V $\to$ A	purpose	-t +cí
V $\to$ N	actor	+el, -t +č
V $\to$ N	nominalization	-t +ní, -at +ání, -it +ení, -it +ění, -out +utí, -ovat +ace
V $\to$ V	imperfectivization	-at +ávat, -it +ovat
V $\to$ V	perfectivization	-at +nout, do+, na+, o+, od+, po+, pro+, pře+, při+, roz+, u+, vy+, z+, za+

Table 2. Table 2: Comparison of different clustering methods on differences of normalized and non-normalized word-vectors trained on Czech National Corpus and clustering into 21 clusters. The results are ordered according to V-measure.

normalized:
Method	cls	H	C	V	A
kmeans	$9.0$	$67.765 260 105 398 81$	$56.443 334 426 537 58$	$61.588 286 270 268 945$	$77.0$
agg	$10.0$	$62.298 836 149 144 83$	$52.878 832 673 545 05$	$57.203 618 829 247 525$	$72.81$
agg (cos)	$8.0$	$38.903 209 662 421 48$	$63.057 527 831 385 315$	$48.119 311 144 985 93$	$47.48$
not normalized:
agg (cos)	$8.0$	$37.933 078 484 721 67$	$64.966 790 802 767 59$	$47.898 804 760 124 25$	$46.38$
agg	$9.0$	$41.189 518 199 886 13$	$39.918 413 606 404 63$	$40.544 005 675 757 084$	$50.09$
kmeans	$7.0$	$39.921 645 643 333 66$	$37.380 220 837 649 69$	$38.609 156 499 922 71$	$46.92$

Table 3. Table 3: Effect of number of clusters with K-means (averaged over 10 runs). inline,]Nebylo by tohle lepsi v grafu? T: mozna bylo, ale v tom pripade by to asi chtelo vybrat jednu hodnotu, kterou budeme plotovat;

model	clust.	H	C	V	A
baseline	$15.0$	$3.788 584 641 751 495 7$	$2.699 600 023 850 02$	$3.152 703 606 529 995$	$30.824 544 582 933 846$
word2vec	$15.0$	$75.981 228 948 974 73$	$57.819 695 135 608 62$	$65.659 852 192 119 53$	$83.058 485 139 022 05$
baseline	$20.0$	$5.121 443 196 628 589$	$3.299 870 732 147 336$	$4.013 648 143 829 394$	$31.323 106 423 777 563$
word2vec	$20.0$	$77.002 694 221 007 18$	$54.035 037 553 372 74$	$63.497 566 854 881 654$	$84.256 951 102 588 7$
baseline	$21.0$	$5.307 073 913 852 612$	$3.367 668 235 616 616$	$4.120 571 069 524 643$	$30.872 483 221 476 51$
word2vec	$21.0$	$77.497 274 615 468 04$	$53.174 248 000 834 82$	$63.066 854 463 208 15$	$84.122 722 914 669 21$
baseline	$22.0$	$5.485 925 790 376 42$	$3.428 593 891 314 421 6$	$4.219 859 313 900 697$	$30.977 948 226 270 37$
word2vec	$22.0$	$77.072 118 543 781 81$	$52.150 460 801 816 32$	$62.196 469 979 527 01$	$83.969 319 271 332 7$
baseline	$25.0$	$6.134 339 714 388 764$	$3.681 503 174 374 864$	$4.601 455 972 279 099 5$	$31.409 395 973 154 364$
word2vec	$25.0$	$80.200 585 892 620 55$	$53.105 652 707 244 57$	$63.894 919 844 539 65$	$87.372 962 607 861 94$

Table 4. Table 4: Results on vectors learned by the NMT models compared to word2vec. K-means clustering with 21 clusters. The results are averaged over 10 independent runs.

model	cls	H	C	V	A
word2vec	$7.9$	$77.532 528 200 633 17$	$53.704 643 556 378 26$	$63.448 168 513 967 22$	$84.180 249 280 920 42$
RNN dec.	$6.8$	$73.090 037 843 784 5$	$52.195 508 167 888 285$	$60.890 011 173 241$	$83.700 862 895 493 77$
RNN+a enc.	$6.4$	$59.443 159 467 697 924$	$44.924 077 537 058 28$	$51.142 425 701 794 73$	$76.097 794 822 627 04$
Transf. enc.	$6.4$	$60.300 659 089 841 87$	$44.235 085 527 062 196$	$51.021 674 265 045 83$	$78.293 384 467 881 1$
RNN+a dec.	$6.8$	$60.937 244 146 566 95$	$40.254 165 534 343 11$	$48.479 806 437 089 586$	$76.404 602 109 300 09$
RNN enc.	$6.4$	$51.903 572 492 760 45$	$45.126 446 401 164 77$	$48.254 949 043 887 97$	$70.488 974 113 135 19$
Transf. dec.	$5.5$	$44.206 949 818 049 43$	$30.562 017 766 880 622$	$36.135 661 669 520 125$	$63.413 231 064 237 78$
baseline	$2.8$	$5.371 408 837 812 099$	$3.407 924 621 692 009$	$4.170 100 755 467 194$	$31.150 527 325 023 965$
POS baseline	$8$	$52.63$	$100.00$	$68.97$	$45.83$

Table 5. Table 5: Precision and recall for the derivation classes. We sampled 250 examples for each class from the data and clustered them with K-means on word2vec embeddings trained on the ČNK. Results presented here are averaged over 5 runs.

Derivation class	precision	recall
ability	$70.713 699 753 896 63$	$68.96$
acting	$47.985 120 892 746 44$	$61.920 000 000 000 01$
actor	$63.025 210 084 033 62$	$60.0$
adjective $\to$ adverb	$87.719 298 245 614 04$	$28.0$
designation	$50.0$	$66.32$
diminutization	$24.090 909 090 909 09$	$42.4$
feature	$81.287 044 877 222 7$	$76.8$
general property	$63.215 061 549 601 74$	$69.84$
imperfectivization	$96.995 305 164 319 25$	$82.64$
instrument / scientist	$97.458 563 535 911 61$	$70.56$
man $\to$ woman	$98.007 471 980 074 73$	$62.96$
man $\to$ woman / diminutization	$34.879 624 192 601 29$	$47.52$
nominalization	$65.388 213 283 442 47$	$55.920 000 000 000 01$
noun $\to$ verb	$96.706 586 826 347 3$	$77.52$
patient	$47.200 000 000 000 01$	$51.920 000 000 000 01$
perfectivization	$92.849 162 011 173 18$	$66.48$
pertaining to	$36.845 039 018 952 065$	$52.879 999 999 999 995$
possessive	$65.287 049 399 198 92$	$78.24$
purpose	$52.330 226 364 846 865$	$31.439 999 999 999 998$
subject	$69.545 154 911 008 58$	$84.4$
super	$31.976 744 186 046 51$	$26.4$

Peer Reviews

No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.

Videos

No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.

Full text

Derivational Morphological Relations in Word Embeddings

Tomáš Musil

Jonáš Vidra

David Mareček

Charles University, Faculty of Mathematics and Physics

Institute of Formal and Applied Linguistics

Malostranské náměstí 25, 118 00 Prague, Czech Republic

{musil,vidra,marecek}@ufal.mff.cuni.cz

Abstract

Derivation is a type of a word-formation process which creates new words from existing ones by adding, changing or deleting affixes. In this paper, we explore the potential of word embeddings to identify properties of word derivations in the morphologically rich Czech language. We extract derivational relations between pairs of words from DeriNet, a Czech lexical network, which organizes almost one million Czech lemmata into derivational trees. For each such pair, we compute the difference of the embeddings of the two words, and perform unsupervised clustering of the resulting vectors. Our results show that these clusters largely match manually annotated semantic categories of the derivational relations (e.g. the relation ‘bake–baker’ belongs to category ‘actor’, and a correct clustering puts it into the same cluster as ‘govern–governor’).

1 Introduction

Word embeddings are a way of representing discrete words in a continuous space. Embeddings are used in neural networks trained for various tasks, e.g. in neural machine translation (NMT), or can be pre-trained in various versions of language models to be used as continuous representations of words for other tasks. One of the most popular frameworks for training word embeddings is word2vec (Mikolov et al., 2013).

In this paper, we examine whether the word embeddings (trained on the whole words, not using any subword units or individual characters) capture derivational relations. We do this to better understand what different neural networks represent about words and to provide a base for further development of derivational networks.

inline,]vyřešit opakování

Derivation is a type of word-formation process which creates new words from existing ones by adding, changing or deleting affixes. For example, the word “collide” can be used as a base for deriving e.g. the words “collider” or “collision”. The derived word “collision” can be, in turn, used as a base for “collisional”.

Words derived from a single root create derivational families, which can be approximated by directed acyclic graphs or (with some loss of information) trees; see Figure 1 for an example.

Derivational relations have two sides: form-based and semantic. For a pair of words to be considered derivationally related, the two words must be related both by their phonological or orthographical forms and by their meaning.

inline,]In this paper we explore the relation between the formal and semantical side of derivations on Czech data.

2 Related work

We have not found any prior work aimed specifically at derivational relations in word embeddings.

Cotterell and Schütze (2018) present a model of the semantics and structure of derivationally complex words. Our work differs in that we are examining how are derivational relations represented in preexisting applications.

Gladkova et al. (2016) detect morphological and semantic relations (including some derivational relations) with word embeddings. Their approach is analogy-based and they conclude that their “experiments show that derivational and lexicographic relations remain a major challenge”.

Gábor et al. (2017) explore vector spaces for semantic relations, using unsupervised clustering. They evaluate the clustering on 9 semantic relation classes. Our approach is similar, but we focus on derivational relations.

Soricut and Och (2015) use word embeddings to induce morphological segmentation in an unsupervised manner. Some of the relations between words that this approach implicitly uses are derivational.

3 Data

In this section, we describe the network of derivational relations and the corpora used in our experiments.

3.1 DeriNet

There are several large networks of derivational relations available for use in research, e.g. CELEX for Dutch, English and German (Baayen et al., 1995), Démonette for French (Hathout and Namer, 2014), DeriNet for Czech (Ševčíková and Žabokrtský, 2014) or DErivBase for German (Zeller et al., 2014). A more complete listing was published by Kyjánek (2018).

For our research, we chose to use the DeriNet-1.6 network mainly due to its large size – with over a million lemmata (citation forms), it is over three times larger than the second largest resource listed by Kyjánek (2018), DErivBase with 280,336 lemmata. Also, the authors are native speakers of Czech, which was necessary for the annotation of derivation classes (see Section 4 below). Large corpora are available for Czech (Bojar et al., 2016; Hnátková et al., 2014), which we need for training the word embeddings.

DeriNet is a network which approximates derivational families using trees – the lemmata it contains are annotated with a single derivational parent or nothing in case the word is either not derived or a parent has not been assigned yet. It contains 1,025,095 lemmata connected by 803,404 relations.

There is a fine line between derivation and inflection and in general, these processes are hard to separate from each other (see e.g. ten Hacken, 2014). Both change base words using affixes, but they differ in the type of the outcome: derivation creates new words, inflection only creates forms of the base word. DeriNet differentiates derivation from inflection the same way the Czech morphological tool MorphoDiTa (Straková et al., 2014) does – it considers the processes handled by the MorphoDiTa tool to be inflectional and other affixations derivational. This is in line with the Czech linguistic tradition (Dokulil et al., 1986), except perhaps for the handling of the two main borderline cases, whose categorization varies: negation (considered inflectional by us) and verbal aspect changes (considered derivational).

3.2 Word Embeddings

In our experiments, we compare the word embeddings obtained by the standard word2vec skip-gram model (Mikolov et al., 2013) with word embeddings learned when training three different neural machine translation (NMT) models (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017). The size of word embeddings is 512 for all the models.

NMT models are trained between English and Czech in both directions. We use the CzEng 1.6 parallel corpus (Bojar et al., 2016), section c-fiction (78 million tokens) and the Neural Monkey toolkit (Helcl et al., 2018)111https://github.com/ufal/neuralmonkey for training the models. We experiment with three architectures:

•

RNN: a simple recurrent neural (RNN) architecture (Sutskever et al., 2014) without attention mechanism, LSTM size 1,024

•

RNN+a: RNN architecture with attention mechanism (Bahdanau et al., 2015), and

•

Transf.: the Transformer (big) architecture (Vaswani et al., 2017) with 6 layers, hidden size 4,096 and 16 attention heads.

Unlike the standard setting in which embeddings of the source and the target words are shared in a common vector space, we use two separated dictionaries (each containing 25,000 word forms). We also do not use any kind of sub-word units. By this setting, we assure that the word vectors are not influenced by any other words that do not belong to the examined language. We extract the encoder word-embeddings from Czech-English NMT model and the decoder word-embeddings from the English-Czech model.

The word2vec system is trained on the Czech National Corpus (Hnátková et al., 2014), version syn 4, which has 4.6 billion tokens. It is a common practice (Mikolov et al., 2013) to normalize the resulting vectors, so that the length of each vector is equal to 1. We report results for both normalized and non-normalized vectors. In order to compare word2vec model with NMT models, we also train word2vec on the Czech part of the data used for training the NMT models.

All the word embeddings are trained on the word forms.

To assign an embedding to the lemma from DeriNet, we simply select the embedding of the word form which is the same as the given lemma.

4 Annotation of Derivational Relations

The derivational relations in DeriNet are not labelled in any way. In this section, we describe a simple method of automatic division of relations into derivation types according to changes in prefixes and suffixes and then manual merging of these types into derivation classes.

When assigning a derivation type to a relation, we first identify the longest common substring of the two related words. For instance, for the relation “padat $\to$ padnout”, the longest common substring is “pad”. Then, we identify prefixes and suffixes using the ‘+’ sign for addition and ‘-’ sign for deletion. A sign after the string indicates a prefix and a sign before the string indicates a suffix. Our example “padat $\to$ padnout” would therefore belong to the derivation type “-at +nout”, which means deleting the suffix “at” and adding the suffix “nout”. Derivation type “na+” means to add the prefix “na”, etc.

When applied on the DeriNet relations, we identified 5,371 derivation types in total. We selected only 71 most frequent types (only those that have at least 250 instances in DeriNet).222We count only such relations, for which both the lemmata occur at least 5 times in the Czech National Corpus. After that, two annotators333The annotators are both native speakers of Czech and they worked together in one shared document. manually merged the 71 derivation types into 21 classes. The classes of derivations are listed in Table 1. The class super+ contains derivations from nouns to nouns and from adjectives to adjectives. Except for insignificant noise in the data, each of the rest of the classes contain only derivations for one POS pair.

The classes were designed in a way to separate different meanings of derivations where possible, and keep different types with the same meaning together (e.g. ‘+ová’ and ‘-a +ová’, which derive feminine surnames).

5 Unsupervised Clustering

We want to know whether and how the derivational relations are captured in the embedding space. We hypothetize that in that case the differences between embedding vectors for the words in a derivational relation would cluster according to the classes we defined.

We perform unsupervised clustering of such differences using three algorithms: inline,]proc nefunguje cosine distance?

•

kmeans: K-means algorithm MacQueen (1967),444We used standard Euclidean distance. The cosine distance does not work at all.

•

agg: Hierarchical agglomerative clustering using Euclidean distance and Ward’s linkage criterion Joe H. Ward (1963),555We experiment also with other linking criteria, however, they performed much worse compared to the Ward’s criterion.

•

agg (cos): The same hierarchical agglomerative clustering, but using cosine distance instead of Euclidean.

For each word pair $W_{1}$ and $W_{2}$ , where $W_{1}$ is the derivational parent of $W_{2}$ and their embeddings $v_{1}$ and $v_{2}$ , the clustering algorithm only gets the differience vector $d=v_{2}-v_{1}$ . The information about the word forms and their derivation type is only used in evaluation.

We evaluate the clustering quality by homogeneity (H), completeness (C) and V-measure (V) (Rosenberg and Hirschberg, 2007). These are entropy based methods, which can be compared across any number of clusters. Homogeneity is a measure of the ratio of instances of a single class pertaining to a single cluster. Completeness measures the ratio of the member of a given class that is assigned to the same cluster. inline,]není na to jednoduchá rovnice jako definice nebo citace někoho, kdo to takto používá (třeba Gábor)? V-measure is computed as the harmonic mean of homogeneity and completeness scores.

Following Gábor et al. (2017), we also report the accuracy (A) that would be achieved by the clustering if we assigned every cluster to the class that is most frequent in this cluster and then used the clustering as a classifier. The number of classes (cls) shows how many classes were assigned to at least one of the clusters.

6 Results

The results on the vectors trained on Czech National Corpus and comparison of normalized and non-normalized versions are summarized in Table 2. We can see that the normalization helps both clustering methods significantly. inline,]The effect of normalization is small when using the cosine distance. Co to znamena? Nemela by byt po normalizaci spis kosinova podobna te euklidovske? The best method, i.e. the K-means used on the normalized word vectors is used in the next experiments.

In Table 3, we examine the effect of the number of clusters on the clustering quality. We compare our models to the baseline, in which each derivation pair is assigned to a random cluster. The table shows that regardless of the number of clusters, the clustering on the word2vec embeddings performs better than the random baseline. It shows that as we allow the K-means algorithm to form more clusters, the homogeneity increases and the completeness decreases. The V-measure is highest with the lowest number of clusters. This may be because the clusters are of uneven size. The accuracy on the word2vec model embeddings is highest around the number of clusters that corresponds to the number of classes in the data.

Table 4 presents the results of clustering the differences of embedding vectors from NMT models. The cls columns shows how many different classes are assigned. Because some classes are more frequent than others, they may form the majority in multiple clusters. This is why random baseline assignes less than 3 different classes on average. We see that word2vec (trained on the Czech side of the parallel corpus) captures more information about derivations than NMT models. RNN models store more information in the embeddings if they do not utilize the attention mechanism. Even less information is stored in the embeddings by the Transformer architecture. This is probably because while in attention-less model the embedding is the only set of parameters directly associated with the given word, in the attention model the information can be split between embeddings and the attention weights. The transformer architecture with residual connections has even more parameters associated with a given word. Decoder in general stores more information about relation between words in the embeddings than encoder, presumably because it partially supplies the role of a language model.

We also evaluated clustering by POS tags (POS baseline in Table 4), where we created 8 clusters based on the POS tags of the parent and child words in the derivational relation. This clustering has a high V-measure, because its completeness is 100 % (the super+ class is not present in the NMT data and for all the other classes it holds that each member of a class has the same parent-child POS tags pair). But it has lower accuracy than all the other models (except for the random baseline), showing that the unsupervised clustering does more than just clustering by POS.

The data naturally contains classes with significant differences in size. To prevent the small classes from being underrepresented, we also evaluated the clustering on a dataset, where the same number of derivation pairs was sampled from each class. Results for the experiment with classes of the same size are listed in Table 5. The results show that the classification does not rely only on changes of part-of-speech. Both imperfectization and perfectivization classses are classified well (97 % precision, 83 % recall and 93 % precision, 66 % recall respectively), even though they are both derivation from verbs to verbs. The only classes that have both precision and recall under 50 % are those being confused with diminutization: man $\to$ woman shares one common derivation type with diminutization, and the class super, which contains only the prefix “super” and is therefore opposite to diminutization, sharing the same semantic axis.

7 Conclusion

Our results show that word-level word embeddings capture information about semantic classes of derivational relations between words, despite not having any information about the orthography or morphological makeup of the words, and therefore not knowing about the formal relation between the words.

It is possible to cluster differences between embeddings in derivational relations, and the assigned clusters correspond to the semantic classes of the relations. The word2vec embeddings generally result in a better clustering than embeddings from the NMT models, and embeddings from the decoder of a plain RNN model perform better than those from NMT models with attention. All these methods outperform a random-assignment clustering baseline and POS clustering baseline.

inline,]future work: víc jazyků

inline,color=blue!40,]Chceme tohle zmiňovat? Když tak to smažte.

A possible extension of this work would compare the word-level word embeddings with character-level or subword embeddingsexplain? which could capture the meaning of individual morphemes. Our preliminary research (not reported here) suggests that FastText embeddings grave2018fasttext, which utilize subword units, outperform the word-level embeddings.

Acknowledgments

This work has been supported by the grant 18-02196S of the Czech Science Foundation. This study was supported by the Charles University Grant Agency (project No. 1176219). This research was partially supported by SVV project number 260 453. This work has been using language resources and tools developed, stored and distributed by the LINDAT/CLARIN project of the Ministry of Education, Youth and Sports of the Czech Republic (project LM2015071).

Bibliography22

The reference list from the paper itself. Each links out to its DOI / PubMed record.

1Baayen et al. (1995) R. Harald Baayen, Richard Piepenbrock, and Leon Gulikers. 1995. The CELEX lexical database (CD-ROM).
2Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate . In 3rd International Conference on Learning Representations, ICLR 2015, San Diego .
3Bojar et al. (2016) Ondřej Bojar, Ondřej Dušek, Tom Kocmi, Jindřich Libovický, Michal Novák, Martin Popel, Roman Sudarikov, and Dušan Variš. 2016. Cz Eng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered. In Text, Speech, and Dialogue: 19th International Conference, TSD 2016 , number 9924 in Lecture Notes in Computer Science, pages 231–238, Cham / Heidelberg / New York / Dordrecht / London. Masaryk University, Springer International Publishing.
4Cotterell and Schütze (2018) Ryan Cotterell and Hinrich Schütze. 2018. Joint semantic synthesis and morphological analysis of the derived word. Transactions of the Association of Computational Linguistics , 6:33–48.
5Dokulil et al. (1986) Miloš Dokulil, Karel Horálek, Jiřina Hůrková, Miloslava Knappová, Jan Petr, and others. 1986. Mluvnice češtiny (1) , 1 edition. Academia, Prague, Czech Republic.
6Gábor et al. (2017) Kata Gábor, Haïfa Zargayouna, Isabelle Tellier, Davide Buscaldi, and Thierry Charnois. 2017. Exploring vector spaces for semantic relations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 1814–1823.
7Gladkova et al. (2016) Anna Gladkova, Aleksandr Drozd, and Satoshi Matsuoka. 2016. Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t. In Proceedings of the NAACL Student Research Workshop , pages 8–15.
8ten Hacken (2014) Pius ten Hacken. 2014. The Oxford Handbook of Derivational Morphology , Oxford Handbooks in Linguistics, chapter Delineating Derivation and Inflection. Oxford University Press, Oxford, United Kingdom.