Can Large Language Models Generate High-quality Patent Claims?

TL;DR

This study evaluates the ability of large language models to generate high-quality patent claims, revealing strengths in initial claim creation and highlighting the need for domain-specific models and further refinement.

Contribution

It constructs a new dataset for patent claim generation and compares general and patent-specific LLMs, showing GPT-4's superior performance and identifying areas for improvement.

Findings

GPT-4 outperforms other LLMs in patent claim quality

Current patent-specific LLMs underperform compared to general models

Fine-tuning improves claim completeness and clarity

Abstract

Large language models (LLMs) have shown exceptional performance across various text generation tasks but remain under-explored in the patent domain, which offers highly structured and precise language. This paper constructs a dataset to investigate the performance of current LLMs in patent claim generation. Our results demonstrate that generating claims based on patent descriptions outperforms previous research relying on abstracts. Interestingly, current patent-specific LLMs perform much worse than state-of-the-art general LLMs, highlighting the necessity for future research on in-domain LLMs. We also find that LLMs can produce high-quality first independent claims, but their performances markedly decrease for subsequent dependent claims. Moreover, fine-tuning can enhance the completeness of inventions' features, conceptual clarity, and feature linkage. Among the tested LLMs, GPT-4 demonstrates the best performance in comprehensive human evaluations by patent experts, with better feature coverage, conceptual clarity, and technical coherence. Despite these capabilities, comprehensive revision and modification are still necessary to pass rigorous patent scrutiny and ensure legal robustness.