Explainable Compliance Detection with Multi-Hop Natural Language Inference on Assurance Case Structure

TL;DR

This paper introduces EXCLAIM, a multi-hop natural language inference approach for explainable compliance detection in assurance cases, leveraging LLMs to generate data and improve regulatory compliance automation.

Contribution

It presents a novel multi-hop NLI framework for compliance detection, addressing data scarcity with LLM-generated assurance cases and introducing new metrics for evaluation.

Findings

Effective multi-hop inference on GDPR assurance cases

Improved explainability and traceability in compliance detection

Potential to automate complex regulatory validation processes

Abstract

Ensuring complex systems meet regulations typically requires checking the validity of assurance cases through a claim-argument-evidence framework. Some challenges in this process include the complicated nature of legal and technical texts, the need for model explanations, and limited access to assurance case data. We propose a compliance detection approach based on Natural Language Inference (NLI): EXplainable CompLiance detection with Argumentative Inference of Multi-hop reasoning (EXCLAIM). We formulate the claim-argument-evidence structure of an assurance case as a multi-hop inference for explainable and traceable compliance detection. We address the limited number of assurance cases by generating them using large language models (LLMs). We introduce metrics that measure the coverage and structural consistency. We demonstrate the effectiveness of the generated assurance case from GDPR requirements in a multi-hop inference task as a case study. Our results highlight the potential of NLI-based approaches in automating the regulatory compliance process.