From Policy to Logic for Efficient and Interpretable Coverage Assessment

TL;DR

This paper presents a hybrid approach combining retrieval and symbolic reasoning to improve efficiency and interpretability in policy coverage assessment, reducing inference costs and enhancing accuracy.

Contribution

It introduces a novel hybrid system that pairs retrieval with symbolic reasoning to support human policy reviewers more effectively.

Findings

44% reduction in inference cost

4.5% improvement in F1 score

Enhanced interpretability of policy analysis

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in interpreting lengthy, complex legal and policy language. However, their reliability can be undermined by hallucinations and inconsistencies, particularly when analyzing subjective and nuanced documents. These challenges are especially critical in medical coverage policy review, where human experts must be able to rely on accurate information. In this paper, we present an approach designed to support human reviewers by making policy interpretation more efficient and interpretable. We introduce a methodology that pairs a coverage-aware retriever with symbolic rule-based reasoning to surface relevant policy language, organize it into explicit facts and rules, and generate auditable rationales. This hybrid system minimizes the number of LLM inferences required which reduces overall model cost. Notably, our approach achieves a 44% reduction in inference cost alongside a 4.5% improvement in F1 score, demonstrating both efficiency and effectiveness.