CSDR-BERT: a pre-trained scientific dataset match model for Chinese Scientific Dataset Retrieval

TL;DR

This paper introduces CSDR-BERT, a specialized pre-trained model for retrieving Chinese scientific datasets, leveraging enhanced BERT-based techniques and demonstrating superior performance over existing models.

Contribution

The study develops a novel Chinese scientific dataset retrieval model by integrating Sentence-BERT, SimCSE, and KNN, optimized with cosent loss, and validates its effectiveness through extensive experiments.

Findings

Outperforms competing models on public and self-built datasets.

Validates the effectiveness of pre-training techniques for Chinese scientific dataset retrieval.

Demonstrates improved semantic matching accuracy in scientific IR tasks.

Abstract

As the number of open and shared scientific datasets on the Internet increases under the open science movement, efficiently retrieving these datasets is a crucial task in information retrieval (IR) research. In recent years, the development of large models, particularly the pre-training and fine-tuning paradigm, which involves pre-training on large models and fine-tuning on downstream tasks, has provided new solutions for IR match tasks. In this study, we use the original BERT token in the embedding layer, improve the Sentence-BERT model structure in the model layer by introducing the SimCSE and K-Nearest Neighbors method, and use the cosent loss function in the optimization phase to optimize the target output. Our experimental results show that our model outperforms other competing models on both public and self-built datasets through comparative experiments and ablation implementations. This study explores and validates the feasibility and efficiency of pre-training techniques for semantic retrieval of Chinese scientific datasets.