Benchmarking and Enabling Efficient Chinese Medical Retrieval via Asymmetric Encoders

TL;DR

This paper introduces a new benchmark and an asymmetric retrieval model for Chinese medical text retrieval, achieving high accuracy with low latency suitable for real-time applications.

Contribution

It presents CMedTEB, a high-quality Chinese medical text benchmark, and CARE, an asymmetric encoder architecture with a novel training strategy, advancing retrieval performance.

Findings

CARE outperforms state-of-the-art symmetric models on CMedTEB.

The benchmark CMedTEB is curated with expert validation to ensure high quality.

CARE achieves superior retrieval accuracy without increasing inference latency.

Abstract

Effective medical text retrieval requires both high accuracy and low latency. While LLM-based embedding models possess powerful retrieval capabilities, their prohibitive latency and high computational cost limit their application in real-time scenarios. Furthermore, the lack of comprehensive and high-fidelity benchmarks hinders progress in Chinese medical text retrieval. In this work, we introduce the Chinese Medical Text Embedding Benchmark (CMedTEB), a benchmark spanning three kinds of practical embedding tasks: retrieval, reranking, and semantic textual similarity (STS). Distinct from purely automated datasets, CMedTEB is curated via a rigorous multi-LLM voting pipeline validated by clinical experts, ensuring gold-standard label quality while effectively mitigating annotation noise. On this foundation, we propose the Chinese Medical Asymmetric REtriever (CARE), an asymmetric architecture that pairs a lightweight BERT-style encoder for online query encoding with a powerful LLM-based encoder for offline document encoding. However, optimizing such an asymmetric retriever with two structurally different encoders presents distinctive challenges. To address this, we introduce a novel two-stage training strategy that progressively bridges the query and document representations. Extensive experiments demonstrate that CARE surpasses state-of-the-art symmetric models on CMedTEB, achieving superior retrieval performance without increasing inference latency.