Towards Efficient and Explainable Hate Speech Detection via Model Distillation

TL;DR

This paper introduces a method to distill large language models into smaller, efficient models that can accurately classify and explain hate speech, making detection more accessible and interpretable.

Contribution

The paper presents a novel distillation approach using Chain-of-Thought to produce smaller models that maintain explanation quality and improve classification performance.

Findings

Distilled models match large models in explanation quality.

Distilled models outperform large models in classification accuracy.

Smaller models are more suitable for operational deployment.

Abstract

Automatic detection of hate and abusive language is essential to combat its online spread. Moreover, recognising and explaining hate speech serves to educate people about its negative effects. However, most current detection models operate as black boxes, lacking interpretability and explainability. In this context, Large Language Models (LLMs) have proven effective for hate speech detection and to promote interpretability. Nevertheless, they are computationally costly to run. In this work, we propose distilling big language models by using Chain-of-Thought to extract explanations that support the hate speech classification task. Having small language models for these tasks will contribute to their use in operational settings. In this paper, we demonstrate that distilled models deliver explanations of the same quality as larger models while surpassing them in classification performance. This dual capability, classifying and explaining, advances hate speech detection making it more affordable, understandable and actionable.