FDA AI Search: Making FDA-Authorized AI Devices Searchable

TL;DR

FDA AI Search is a semantic search tool that helps healthcare providers and developers find FDA-authorized AI medical devices more effectively by leveraging large language models and embedding-based retrieval.

Contribution

The paper introduces a novel semantic search system for FDA device data, improving over keyword-based methods with LLM-extracted features and embedding retrieval.

Findings

Quantitative evaluation shows improved retrieval accuracy.

Qualitative analysis confirms relevance of search results.

The system aids clinical decision-making and AI device development.

Abstract

Over 1,200 AI-enabled medical devices have received marketing authorization from the U.S. FDA, yet identifying devices suited to specific clinical needs remains challenging because the FDA's databases contain only limited metadata and non-searchable summary PDFs. To address this gap, we developed FDA AI Search, a website that enables semantic querying of FDA-authorized AI-enabled devices. The backend includes an embedding-based retrieval system, where LLM-extracted features from authorization summaries are compared to user queries to find relevant matches. We present quantitative and qualitative evaluation that support the effectiveness of the retrieval algorithm compared to keyword-based methods. As FDA-authorized AI devices become increasingly prevalent and their use cases expand, we envision that the tool will assist healthcare providers in identifying devices aligned with their clinical needs and support developers in formulating novel AI applications.