Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks

TL;DR

This paper introduces a multi-attention recurrent network model that improves patent citation forecasting by leveraging multiple historical citation sequences, including assignee and inventor data, and predicts both citation timing and category.

Contribution

It presents a novel sequence-to-sequence model with an attention-of-attention mechanism that captures dependencies across multiple citation-related sequences for enhanced forecasting.

Findings

Outperforms existing models in citation prediction accuracy

Effectively forecasts both citation timing and categories

Utilizes large USPTO dataset for validation

Abstract

Modeling and forecasting forward citations to a patent is a central task for the discovery of emerging technologies and for measuring the pulse of inventive progress. Conventional methods for forecasting these forward citations cast the problem as analysis of temporal point processes which rely on the conditional intensity of previously received citations. Recent approaches model the conditional intensity as a chain of recurrent neural networks to capture memory dependency in hopes of reducing the restrictions of the parametric form of the intensity function. For the problem of patent citations, we observe that forecasting a patent's chain of citations benefits from not only the patent's history itself but also from the historical citations of assignees and inventors associated with that patent. In this paper, we propose a sequence-to-sequence model which employs an attention-of-attention mechanism to capture the dependencies of these multiple time sequences. Furthermore, the proposed model is able to forecast both the timestamp and the category of a patent's next citation. Extensive experiments on a large patent citation dataset collected from USPTO demonstrate that the proposed model outperforms state-of-the-art models at forward citation forecasting.