PromptGate Client Adaptive Vision Language Gating for Open Set Federated Active Learning

TL;DR

PromptGate is a federated active learning framework that uses adaptive vision-language gating to improve data efficiency and OOD detection in privacy-sensitive medical imaging applications.

Contribution

It introduces a novel federated prompt optimization method that enhances open-set active learning by dynamically filtering out OOD samples without sharing patient data.

Findings

Maintains over 95% ID purity in open-set federated learning.

Achieves 98% OOD recall in medical imaging benchmarks.

Outperforms static prompting methods in open-set scenarios.

Abstract

Deploying medical AI across resource-constrained institutions demands data-efficient learning pipelines that respect patient privacy. Federated Learning (FL) enables collaborative medical AI without centralising data, yet real-world clinical pools are inherently open-set, containing out-of-distribution (OOD) noise such as imaging artifacts and wrong modalities. Standard Active Learning (AL) query strategies mistake this noise for informative samples, wasting scarce annotation budgets. We propose PromptGate, a dynamic VLM-gated framework for Open-Set Federated AL that purifies unlabeled pools before querying. PromptGate introduces a federated Class-Specific Context Optimization: lightweight, learnable prompt vectors that adapt a frozen BiomedCLIP backbone to local clinical domains and aggregate globally via FedAvg -- without sharing patient data. As new annotations arrive, prompts progressively sharpen the ID/OOD boundary, turning the VLM into a dynamic gatekeeper that is strategy-agnostic: a plug-and-play pre-selection module enhancing any downstream AL strategy. Experiments on distributed dermatology and breast imaging benchmarks show that while static VLM prompting degrades to 50% ID purity, PromptGate maintains $>$95% purity with 98% OOD recall.