Classification of Visualization Types and Perspectives in Patents

TL;DR

This paper applies advanced deep learning techniques to classify visualization types and perspectives in patent images, enhancing patent search and analysis by extending datasets and providing new annotations.

Contribution

It introduces a novel deep learning-based classification approach for patent visualization images, extending existing datasets with new classes and annotations.

Findings

Deep learning methods are feasible for patent image classification.

Extended dataset with ten visualization classes and hierarchical perspective labels.

Source code and datasets will be publicly available.

Abstract

Due to the swift growth of patent applications each year, information and multimedia retrieval approaches that facilitate patent exploration and retrieval are of utmost importance. Different types of visualizations (e.g., graphs, technical drawings) and perspectives (e.g., side view, perspective) are used to visualize details of innovations in patents. The classification of these images enables a more efficient search and allows for further analysis. So far, datasets for image type classification miss some important visualization types for patents. Furthermore, related work does not make use of recent deep learning approaches including transformers. In this paper, we adopt state-of-the-art deep learning methods for the classification of visualization types and perspectives in patent images. We extend the CLEF-IP dataset for image type classification in patents to ten classes and provide manual ground truth annotations. In addition, we derive a set of hierarchical classes from a dataset that provides weakly-labeled data for image perspectives. Experimental results have demonstrated the feasibility of the proposed approaches. Source code, models, and dataset will be made publicly available.