A novel approach to measuring the scope of patent claims based on probabilities obtained from (large) language models

TL;DR

This paper introduces a method to quantify patent claim scope using probabilities from language models, where less probable, more surprising concepts indicate narrower claims, validated across various models including large language models like GPT-2 and davinci-002.

Contribution

It presents a novel information-theoretic approach to measure patent claim scope leveraging language model probabilities, demonstrating the effectiveness of large models over simpler frequency-based methods.

Findings

LLMs outperform frequency-based models in scope measurement

Character count is a more reliable indicator than word count

Simplest models relate scope to the reciprocal of claim length

Abstract

This work proposes to measure the scope of a patent claim as the reciprocal of self-information contained in this claim. Self-information is calculated based on a probability of occurrence of the claim, where this probability is obtained from a language model. Grounded in information theory, this approach is based on the assumption that an unlikely concept is more informative than a usual concept, insofar as it is more surprising. In turn, the more surprising the information required to define the claim, the narrower its scope. Seven language models are considered, ranging from simplest models (each word or character has an identical probability) to intermediate models (based on average word or character frequencies), to large language models (LLMs) such as GPT2 and davinci-002. Remarkably, when using the simplest language models to compute the probabilities, the scope becomes proportional to the reciprocal of the number of words or characters involved in the claim, a metric already used in previous works. Application is made to multiple series of patent claims directed to distinct inventions, where each series consists of claims devised to have a gradually decreasing scope. The performance of the language models is then assessed through several ad hoc tests. The LLMs outperform models based on word and character frequencies, which themselves outdo the simplest models based on word or character counts. Interestingly, however, the character count appears to be a more reliable indicator than the word count.