A deep Natural Language Inference predictor without language-specific training data

TL;DR

This paper introduces a language-agnostic NLP approach for Natural Language Inference that leverages knowledge distillation and machine translation, enabling effective inference without language-specific training data.

Contribution

The paper presents a novel knowledge distillation technique that allows NLI models to operate across languages without language-specific datasets, outperforming translation-based methods.

Findings

The proposed model performs well on multiple translated NLI datasets.

It generalizes across different NLP tasks beyond NLI.

Outperforms traditional machine translation approaches in NLI tasks.

Abstract

In this paper we present a technique of NLP to tackle the problem of inference relation (NLI) between pairs of sentences in a target language of choice without a language-specific training dataset. We exploit a generic translation dataset, manually translated, along with two instances of the same pre-trained model - the first to generate sentence embeddings for the source language, and the second fine-tuned over the target language to mimic the first. This technique is known as Knowledge Distillation. The model has been evaluated over machine translated Stanford NLI test dataset, machine translated Multi-Genre NLI test dataset, and manually translated RTE3-ITA test dataset. We also test the proposed architecture over different tasks to empirically demonstrate the generality of the NLI task. The model has been evaluated over the native Italian ABSITA dataset, on the tasks of Sentiment Analysis, Aspect-Based Sentiment Analysis, and Topic Recognition. We emphasise the generality and exploitability of the Knowledge Distillation technique that outperforms other methodologies based on machine translation, even though the former was not directly trained on the data it was tested over.