Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study

TL;DR

This paper explores whether automated, LLM-generated textual explanations can improve NLP model classification performance, offering a scalable alternative to costly human annotations.

Contribution

It introduces an automated framework using multiple LLMs to generate explanations and evaluates their impact on model performance across benchmark datasets.

Findings

LLM-generated explanations are highly competitive with human annotations.

Automated explanations improve model performance on NLP tasks.

The approach offers a scalable solution for dataset enrichment and interpretability.

Abstract

In the rapidly evolving field of Explainable Natural Language Processing (NLP), textual explanations, i.e., human-like rationales, are pivotal for explaining model predictions and enriching datasets with interpretable labels. Traditional approaches rely on human annotation, which is costly, labor-intensive, and impedes scalability. In this work, we present an automated framework that leverages multiple state-of-the-art large language models (LLMs) to generate high-quality textual explanations. We rigorously assess the quality of these LLM-generated explanations using a comprehensive suite of Natural Language Generation (NLG) metrics. Furthermore, we investigate the downstream impact of these explanations on the performance of pre-trained language models (PLMs) and LLMs across natural language inference tasks on two diverse benchmark datasets. Our experiments demonstrate that automated explanations exhibit highly competitive effectiveness compared to human-annotated explanations in improving model performance. Our findings underscore a promising avenue for scalable, automated LLM-based textual explanation generation for extending NLP datasets and enhancing model performance.