Divide and Conquer: An Ensemble Approach for Hostile Post Detection in Hindi

TL;DR

This paper presents a two-level ensemble system using BERT and statistical classifiers for detecting hostile posts in Hindi, achieving high accuracy and ranking second in a shared task.

Contribution

The paper introduces a novel ensemble architecture combining BERT and statistical classifiers for multi-label hostile post detection in Hindi social media data.

Findings

Achieved 0.9709 F1 score on coarse-grained hostility detection.

Secured 2nd rank out of 45 teams in the shared task.

High potential for further improvement with fine-tuning.

Abstract

Recently the NLP community has started showing interest towards the challenging task of Hostile Post Detection. This paper present our system for Shared Task at Constraint2021 on "Hostile Post Detection in Hindi". The data for this shared task is provided in Hindi Devanagari script which was collected from Twitter and Facebook. It is a multi-label multi-class classification problem where each data instance is annotated into one or more of the five classes: fake, hate, offensive, defamation, and non-hostile. We propose a two level architecture which is made up of BERT based classifiers and statistical classifiers to solve this problem. Our team 'Albatross', scored 0.9709 Coarse grained hostility F1 score measure on Hostile Post Detection in Hindi subtask and secured 2nd rank out of 45 teams for the task. Our submission is ranked 2nd and 3rd out of a total of 156 submissions with Coarse grained hostility F1 score of 0.9709 and 0.9703 respectively. Our fine grained scores are also very encouraging and can be improved with further finetuning. The code is publicly available.