An Accurate Model for Predicting the (Graded) Effect of Context in Word Similarity Based on Bert

TL;DR

This paper presents a new model based on BERT for predicting how context influences human perception of word similarity, achieving top results in SemEval 2020 tasks.

Contribution

It introduces a methodology to analyze contextual effects on word similarity using BERT embeddings, with state-of-the-art performance in multilingual semantic tasks.

Findings

Won 1st place in Finnish language track

Secured 2nd place in English language track

Demonstrated effectiveness of BERT-based distance measures

Abstract

Natural Language Processing (NLP) has been widely used in the semantic analysis in recent years. Our paper mainly discusses a methodology to analyze the effect that context has on human perception of similar words, which is the third task of SemEval 2020. We apply several methods in calculating the distance between two embedding vector generated by Bidirectional Encoder Representation from Transformer (BERT). Our team will_go won the 1st place in Finnish language track of subtask1, the second place in English track of subtask1.