Beyond Flashcards: Designing an Intelligent Assistant for USMLE Mastery and Virtual Tutoring in Medical Education (A Study on Harnessing Chatbot Technology for Personalized Step 1 Prep)

TL;DR

This paper presents an AI-powered chatbot designed to assist medical students in preparing for USMLE Step 1 and similar exams by providing personalized, on-demand, human-like responses to enhance learning and knowledge retention.

Contribution

It introduces a novel AI assistant that leverages generative AI, retrieval-augmented generation, and prompt engineering to deliver contextually relevant support for medical exam preparation.

Findings

Achieved a response quality score of 0.5985 on a reference-based metric.

Enhanced medical exam preparation through personalized, real-time AI assistance.

Utilized RAG and prompt engineering to improve response accuracy and relevance.

Abstract

Traditional medical basic sciences educational approaches follow a one-size-fits-all model, neglecting the diverse learning styles of individual students. I propose an intelligent AI companion which will fill this gap by providing on-the-fly solutions to students' questions in the context of not only USMLE Step 1 but also other similar examinations in other countries, inter alia, PLAB Part 1 in United Kingdom, and NEET (PG) and FMGE in India. I have harnessed Generative AI for dynamic, accurate, human-like responses and for knowledge retention and application. Users were encouraged to employ prompt engineering, in particular, in-context learning, for response optimization and enhancing the model's precision in understanding the intent of the user through the way the query is framed. The implementation of RAG has enhanced the chatbot's ability to combine pre-existing medical knowledge with generative capabilities for efficient and contextually relevant support. Mistral was employed using Python to perform the needed functions. The digital conversational agent was implemented and achieved a score of 0.5985 on a reference-based metric similar to BLEU and ROUGE scores. My approach addresses a critical gap in traditional medical basic sciences education by introducing an intelligent AI companion which specializes in helping medical aspirants with planning and information retention for USMLE Step 1 and other similar exams. Considering the stress that medical aspirants face in studying for the exam and in obtaining spontaneous answers to medical basic sciences queries, especially whose answers are challenging to obtain by searching online, and obviating a student's need to search bulky medical texts or lengthy indices or appendices, I have been able to create a quality assistant capable of producing ad-libitum responses best suited to the user's needs.