Contradiction Detection in Persian Text

TL;DR

This paper presents a hybrid approach combining rule-based and BERT-based deep learning methods for detecting semantic contradictions in Persian texts, achieving high F-measure scores and outperforming existing algorithms.

Contribution

It introduces a novel rule-based system and a BERT-based deep learning model for Persian contradiction detection, along with a hybrid system that improves performance.

Findings

Rule-based system achieves up to 90% F-measure for negation contradictions.

Deep learning BERT model achieves 73% F-measure on Persian contradiction detection.

Hybrid system reaches approximately 80% F-measure, outperforming individual methods.

Abstract

Detection of semantic contradictory sentences is one of the most challenging and fundamental issues for NLP applications such as recognition of textual entailments. Contradiction in this study includes different types of semantic confrontation, such as conflict and antonymy. Due to lack of sufficient data to apply precise machine learning and specifically deep learning methods to Persian and other low resource languages, rule-based approaches that can function similarly to these systems will be of a great interest. Also recently, emergence of new methods such as transfer learning, has opened up the possibility of deep learning for low-resource languages. Considering two above points, in this study, along with a simple rule-base baseline, a novel rule-base system for identifying semantic contradiction along with a Bert base deep contradiction detection system for Persian texts have been introduced. The rule base system has used frequent rule mining method to extract appropriate contradiction rules using a development set. Extracted rules are tested for different categories of contradictory sentences. In this system the maximum f-measure among contradiction categories is obtained for negation about 90% and the average F-measure of system for all classes is about 76% which outperforms other algorithms on Persian texts. On the other hand, because of medium performance of rule base system for some categories of contradiction, we use a Bert base deep learning system using our translated dataset; with average F-measure of 73. Our hybrid system has f-measure of about 80.