Dental CLAIRES: Contrastive LAnguage Image REtrieval Search for Dental Research

TL;DR

Dental CLAIRES is a contrastive learning-based search tool that retrieves dental radiographs based on text queries, improving research efficiency with high accuracy and an interactive interface.

Contribution

We developed Dental CLAIRES, a novel contrastive learning framework for dental image retrieval using clinical text, addressing data annotation challenges and enhancing research tools.

Findings

Achieved hit@3 ratio of 96%

Obtained MRR of 0.82

Provided an interactive GUI for verification

Abstract

Learning about diagnostic features and related clinical information from dental radiographs is important for dental research. However, the lack of expert-annotated data and convenient search tools poses challenges. Our primary objective is to design a search tool that uses a user's query for oral-related research. The proposed framework, Contrastive LAnguage Image REtrieval Search for dental research, Dental CLAIRES, utilizes periapical radiographs and associated clinical details such as periodontal diagnosis, demographic information to retrieve the best-matched images based on the text query. We applied a contrastive representation learning method to find images described by the user's text by maximizing the similarity score of positive pairs (true pairs) and minimizing the score of negative pairs (random pairs). Our model achieved a hit@3 ratio of 96% and a Mean Reciprocal Rank (MRR) of 0.82. We also designed a graphical user interface that allows researchers to verify the model's performance with interactions.