Adaptive Cost-Efficient Evaluation for Reliable Patent Claim Validation

TL;DR

ACE is a hybrid framework that efficiently validates patent claims by selectively employing large language models for complex cases, achieving high accuracy and cost savings.

Contribution

We introduce ACE, a novel adaptive evaluation method that combines entropy-based routing with legal reasoning protocols, improving cost-efficiency and robustness in patent claim validation.

Findings

ACE achieves 94.95% F1 score on benchmark

Reduces operational costs by 78% compared to standalone LLMs

Entropy-based routing transfers effectively to real USPTO rejections

Abstract

Automated validation of patent claims demands zero-defect tolerance, as even a single structural flaw can render a claim legally defective. Existing evaluation paradigms suffer from a rigidity-resource dilemma: lightweight encoders struggle with nuanced legal dependencies, while exhaustive verification via Large Language Models (LLMs) is prohibitively costly. To bridge this gap, we propose ACE (Adaptive Cost-efficient Evaluation), a hybrid framework that uses predictive entropy to route only high-uncertainty claims to an expert LLM. The expert then executes a Chain of Patent Thought (CoPT) protocol grounded in 35 U.S.C. statutory standards, enabling ACE to resolve long-range legal dependencies that encoder-only models fail to capture. On our constructed benchmark, ACE achieves the best F1 among the evaluated methods at 94.95\% while reducing operational costs by 78\% compared to standalone LLM deployments. Crucially, the entropy-based routing threshold transfers directly to authentic USPTO {\S}112(b) rejections without re-calibration, confirming distributional robustness beyond synthetic settings. To facilitate reproducible research, we release ACE-40k, a 40,000-claim benchmark with MPEP-grounded error annotations, alongside ACE-Real112b, a stress-test corpus of 204 genuine Office Action rejections.