Distilling Closed-Source LLM's Knowledge for Locally Stable and Economic Biomedical Entity Linking

TL;DR

This paper introduces RPDR, a framework that combines closed-source and open-source LLMs to enable stable, cost-effective biomedical entity linking by distilling knowledge into open-source models for local deployment.

Contribution

The paper proposes a novel framework, RPDR, that leverages closed-source LLMs to generate training data and fine-tunes open-source LLMs for re-ranking, reducing costs and stability issues.

Findings

RPDR improves accuracy by up to 0.036 on benchmark datasets.

The framework demonstrates effectiveness across multiple languages.

Knowledge distillation enhances local deployability of biomedical LLMs.

Abstract

Biomedical entity linking aims to map nonstandard entities to standard entities in a knowledge base. Traditional supervised methods perform well but require extensive annotated data to transfer, limiting their usage in low-resource scenarios. Large language models (LLMs), especially closed-source LLMs, can address these but risk stability issues and high economic costs: using these models is restricted by commercial companies and brings significant economic costs when dealing with large amounts of data. To address this, we propose ``RPDR'', a framework combining closed-source LLMs and open-source LLMs for re-ranking candidates retrieved by a retriever fine-tuned with a small amount of data. By prompting a closed-source LLM to generate training data from unannotated data and fine-tuning an open-source LLM for re-ranking, we effectively distill the knowledge to the open-source LLM that can be deployed locally, thus avoiding the stability issues and the problem of high economic costs. We evaluate RPDR on two datasets, including one real-world dataset and one publicly available dataset involving two languages: Chinese and English. RPDR achieves 0.019 Acc@1 improvement and 0.036 Acc@1 improvement on the Aier dataset and the Ask A Patient dataset when the amount of training data is not enough. The results demonstrate the superiority and generalizability of the proposed framework.