Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

TL;DR

This paper introduces an interpretable hierarchical attention network model, PatentHAN, for early screening of breakthrough technologies from patent texts, combining predictive accuracy with transparency to aid expert decision-making.

Contribution

The paper presents a novel patent-specific hierarchical attention network that improves interpretability in predicting patent impact, facilitating early identification of breakthrough innovations.

Findings

Effective prediction of future citation counts from patent texts.

Enhanced interpretability through claim-wise attention mechanisms.

Robustness confirmed across different language models and claim types.

Abstract

Despite the usefulness of machine learning approaches for the early screening of potential breakthrough technologies, their practicality is often hindered by opaque models. To address this, we propose an interpretable machine learning approach to predicting future citation counts from patent texts using a patent-specific hierarchical attention network (PatentHAN) model. Central to this approach are (1) a patent-specific pre-trained language model, capturing the meanings of technical words in patent claims, (2) a hierarchical network structure, enabling detailed analysis at the claim level, and (3) a claim-wise self-attention mechanism, revealing pivotal claims during the screening process. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach in early screening of potential breakthrough technologies while ensuring interpretability. Furthermore, we conduct additional analyses using different language models and claim types to examine the robustness of the approach. It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies, providing new insight derived from text mining into technological value.