Digital Socrates: Evaluating LLMs through Explanation Critiques

TL;DR

This paper introduces Digital Socrates, an automatic critique tool for evaluating the quality of explanations generated by large language models, enabling nuanced analysis without human annotation.

Contribution

It defines the new task of explanation critiquing, creates a dataset, and trains an open-source model for automatic, detailed explanation evaluation.

Findings

Digital Socrates reveals insights into model reasoning chains.

It provides high-quality automatic evaluation of explanations.

The tool helps identify and address explanation flaws.

Abstract

While LLMs can provide reasoned explanations along with their answers, the nature and quality of those explanations are still poorly understood. In response, our goal is to define a detailed way of characterizing the explanation capabilities of modern models and to create a nuanced, interpretable explanation evaluation tool that can generate such characterizations automatically, without relying on expensive API calls or human annotations. Our approach is to (a) define the new task of explanation critiquing - identifying and categorizing any main flaw in an explanation and providing suggestions to address the flaw, (b) create a sizeable, human-verified dataset for this task, and (c) train an open-source, automatic critique model (called Digital Socrates) using this data. Through quantitative and qualitative analysis, we demonstrate how Digital Socrates is useful for revealing insights about student models by examining their reasoning chains, and how it can provide high-quality, nuanced, automatic evaluation of those model explanations for the first time. Digital Socrates thus fills an important gap in evaluation tools for understanding and improving the explanation behavior of models.