DesignCLIP: Multimodal Learning with CLIP for Design Patent Understanding

TL;DR

DesignCLIP leverages CLIP-based multimodal learning to improve design patent classification and retrieval by incorporating detailed captions and multi-view image analysis, outperforming existing models.

Contribution

This work introduces DesignCLIP, a novel multimodal framework using CLIP for design patent understanding, with class-aware classification and contrastive learning tailored for patent data.

Findings

Outperforms baseline and SOTA models in patent tasks

Effective in patent classification and retrieval

Enhances multimodal patent analysis

Abstract

In the field of design patent analysis, traditional tasks such as patent classification and patent image retrieval heavily depend on the image data. However, patent images -- typically consisting of sketches with abstract and structural elements of an invention -- often fall short in conveying comprehensive visual context and semantic information. This inadequacy can lead to ambiguities in evaluation during prior art searches. Recent advancements in vision-language models, such as CLIP, offer promising opportunities for more reliable and accurate AI-driven patent analysis. In this work, we leverage CLIP models to develop a unified framework DesignCLIP for design patent applications with a large-scale dataset of U.S. design patents. To address the unique characteristics of patent data, DesignCLIP incorporates class-aware classification and contrastive learning, utilizing generated detailed captions for patent images and multi-views image learning. We validate the effectiveness of DesignCLIP across various downstream tasks, including patent classification and patent retrieval. Additionally, we explore multimodal patent retrieval, which provides the potential to enhance creativity and innovation in design by offering more diverse sources of inspiration. Our experiments show that DesignCLIP consistently outperforms baseline and SOTA models in the patent domain on all tasks. Our findings underscore the promise of multimodal approaches in advancing patent analysis. The codebase is available here: https://anonymous.4open.science/r/PATENTCLIP-4661/README.md.