Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement

TL;DR

This paper introduces Logic-Explainer, a hybrid neuro-symbolic framework that refines and verifies ethical explanations generated by LLMs in NLI tasks, improving their logical validity and interpretability.

Contribution

It presents a novel abductive-deductive approach combining LLMs with external solvers to enhance explanation quality and correctness in ethical NLI.

Findings

Improves explanation accuracy and consistency in ethical NLI tasks

Produces formal proofs supporting model reasoning

Enhances logical validity and alignment of LLM explanations

Abstract

An increasing amount of research in Natural Language Inference (NLI) focuses on the application and evaluation of Large Language Models (LLMs) and their reasoning capabilities. Despite their success, however, LLMs are still prone to factual errors and inconsistencies in their explanations, offering limited control and interpretability for inference in complex domains. In this paper, we focus on ethical NLI, investigating how hybrid neuro-symbolic techniques can enhance the logical validity and alignment of ethical explanations produced by LLMs. Specifically, we present an abductive-deductive framework named Logic-Explainer, which integrates LLMs with an external backward-chaining solver to refine step-wise natural language explanations and jointly verify their correctness, reduce incompleteness and minimise redundancy. An extensive empirical analysis demonstrates that Logic-Explainer can improve explanations generated via in-context learning methods and Chain-of-Thought (CoT) on challenging ethical NLI tasks, while, at the same time, producing formal proofs describing and supporting models' reasoning. As ethical NLI requires commonsense reasoning to identify underlying moral violations, our results suggest the effectiveness of neuro-symbolic methods for multi-step NLI more broadly, opening new opportunities to enhance the logical consistency, reliability, and alignment of LLMs.