BITS Pilani at HinglishEval: Quality Evaluation for Code-Mixed Hinglish Text Using Transformers

TL;DR

This paper presents a method using multi-lingual BERT to evaluate the quality of synthetic Hinglish sentences by measuring their similarity to human-generated sentences, addressing challenges in code-mixed language processing.

Contribution

It introduces a novel approach leveraging multi-lingual BERT for quality assessment of code-mixed Hinglish text, specifically for the HinglishEval task.

Findings

Multi-lingual BERT effectively measures similarity between synthetic and human Hinglish sentences.

The proposed model achieves promising results in the HinglishEval quality evaluation task.

The approach helps improve understanding and processing of code-mixed language data.

Abstract

Code-Mixed text data consists of sentences having words or phrases from more than one language. Most multi-lingual communities worldwide communicate using multiple languages, with English usually one of them. Hinglish is a Code-Mixed text composed of Hindi and English but written in Roman script. This paper aims to determine the factors influencing the quality of Code-Mixed text data generated by the system. For the HinglishEval task, the proposed model uses multi-lingual BERT to find the similarity between synthetically generated and human-generated sentences to predict the quality of synthetically generated Hinglish sentences.