Evaluating Semantic Rationality of a Sentence: A Sememe-Word-Matching Neural Network based on HowNet

TL;DR

This paper introduces SWM-NN, a neural network leveraging HowNet sememes for evaluating sentence semantic rationality, outperforming existing methods by 5.4% in accuracy.

Contribution

The paper presents a novel sememe-based neural network model for semantic rationality evaluation, utilizing fine-grained semantic representations from HowNet.

Findings

SWM-NN outperforms baselines with 5.4% higher accuracy

The model effectively captures semantic dependencies among words

A large-scale dataset was built for evaluation

Abstract

Automatic evaluation of semantic rationality is an important yet challenging task, and current automatic techniques cannot well identify whether a sentence is semantically rational. The methods based on the language model do not measure the sentence by rationality but by commonness. The methods based on the similarity with human written sentences will fail if human-written references are not available. In this paper, we propose a novel model called Sememe-Word-Matching Neural Network (SWM-NN) to tackle semantic rationality evaluation by taking advantage of sememe knowledge base HowNet. The advantage is that our model can utilize a proper combination of sememes to represent the fine-grained semantic meanings of a word within the specific contexts. We use the fine-grained semantic representation to help the model learn the semantic dependency among words. To evaluate the effectiveness of the proposed model, we build a large-scale rationality evaluation dataset. Experimental results on this dataset show that the proposed model outperforms the competitive baselines with a 5.4\% improvement in accuracy.