Research on Evaluation Methods for Patent Novelty Search Systems and Empirical Analysis

TL;DR

This paper introduces a comprehensive evaluation framework for patent novelty search systems, utilizing high-quality datasets and multi-dimensional analysis to improve system performance and reliability.

Contribution

It presents a novel, scalable evaluation methodology that combines dataset construction from citations with multi-faceted analysis metrics for patent search systems.

Findings

Effective in exposing performance differences across scenarios

Provides actionable insights for system improvement

Scalable and practical evaluation framework

Abstract

Patent novelty search systems are critical to IP protection and innovation assessment; their retrieval accuracy directly impacts patent quality. We propose a comprehensive evaluation methodology that builds high-quality, reproducible datasets from examiner citations and X-type citations extracted from technically consistent family patents, and evaluates systems using invention descriptions as inputs. Using Top-k Detection Rate and Recall as core metrics, we further conduct multi-dimensional analyses by language, technical field (IPC), and filing jurisdiction. Experiments show the method effectively exposes performance differences across scenarios and offers actionable evidence for system improvement. The framework is scalable and practical, providing a useful reference for development and optimization of patent novelty search systems