The Locally Deployable Virtual Doctor: LLM Based Human Interface for Automated Anamnesis and Database Conversion

TL;DR

This paper presents MedChat, a fully offline, privacy-preserving AI framework combining an LLM-based medical chatbot and a diffusion-driven avatar for automated clinical anamnesis, suitable for low-resource settings.

Contribution

Introduction of MedChat, a novel locally deployable AI system integrating an LLM chatbot with a realistic avatar for secure, offline medical interviews.

Findings

Achieved stable fine-tuning with strong generalization.

Demonstrated feasibility of fully offline AI for clinical anamnesis.

Ensured complete data separation for privacy and security.

Abstract

Recent advances in large language models made it possible to achieve high conversational performance with substantially reduced computational demands, enabling practical on-site deployment in clinical environments. Such progress allows for local integration of AI systems that uphold strict data protection and patient privacy requirements, yet their secure implementation in medicine necessitates careful consideration of ethical, regulatory, and technical constraints. In this study, we introduce MedChat, a locally deployable virtual physician framework that integrates an LLM-based medical chatbot with a diffusion-driven avatar for automated and structured anamnesis. The chatbot was fine-tuned using a hybrid corpus of real and synthetically generated medical dialogues, while model efficiency was optimized via Low-Rank Adaptation. A secure and isolated database interface was implemented to ensure complete separation between patient data and the inference process. The avatar component was realized through a conditional diffusion model operating in latent space, trained on researcher video datasets and synchronized with mel-frequency audio features for realistic speech and facial animation. Unlike existing cloud-based systems, this work demonstrates the feasibility of a fully offline, locally deployable LLM-diffusion framework for clinical anamnesis. The autoencoder and diffusion networks exhibited smooth convergence, and MedChat achieved stable fine-tuning with strong generalization to unseen data. The proposed system thus provides a privacy-preserving, resource-efficient foundation for AI-assisted clinical anamnesis, also in low-cost settings.