Sememe Prediction for BabelNet Synsets using Multilingual and Multimodal Information

TL;DR

This paper introduces a multimodal, multilingual approach to predict sememes for BabelNet synsets, significantly improving the accuracy of sememe annotation across languages by leveraging synonyms, glosses, and images.

Contribution

It presents a novel multimodal information fusion model for sememe prediction that effectively utilizes BabelNet's rich multilingual and multimodal data, outperforming previous methods.

Findings

Achieved about 10% improvement in MAP and F1 scores over previous methods.

Demonstrated the effectiveness of combining multilingual and multimodal information.

Provided open-source code and data for reproducibility.

Abstract

In linguistics, a sememe is defined as the minimum semantic unit of languages. Sememe knowledge bases (KBs), which are built by manually annotating words with sememes, have been successfully applied to various NLP tasks. However, existing sememe KBs only cover a few languages, which hinders the wide utilization of sememes. To address this issue, the task of sememe prediction for BabelNet synsets (SPBS) is presented, aiming to build a multilingual sememe KB based on BabelNet, a multilingual encyclopedia dictionary. By automatically predicting sememes for a BabelNet synset, the words in many languages in the synset would obtain sememe annotations simultaneously. However, previous SPBS methods have not taken full advantage of the abundant information in BabelNet. In this paper, we utilize the multilingual synonyms, multilingual glosses and images in BabelNet for SPBS. We design a multimodal information fusion model to encode and combine this information for sememe prediction. Experimental results show the substantial outperformance of our model over previous methods (about 10 MAP and F1 scores). All the code and data of this paper can be obtained at https://github.com/thunlp/MSGI.