A Patient-Doctor-NLP-System to contest inequality for less privileged

TL;DR

This paper introduces PDFTEMRA, a compact transformer model designed for accessible medical NLP in low-resource settings, combining distillation, ensemble learning, and random activations to reduce computational costs while maintaining performance.

Contribution

The study presents PDFTEMRA, a novel, resource-efficient transformer architecture tailored for low-resource medical NLP applications, especially aiding visually impaired and low-resource language users.

Findings

Achieves comparable accuracy to state-of-the-art models with lower computational cost.

Effective in medical question-answering and consultation tasks for Hindi and accessibility scenarios.

Demonstrates suitability for resource-constrained healthcare environments.

Abstract

Transfer Learning (TL) has accelerated the rapid development and availability of large language models (LLMs) for mainstream natural language processing (NLP) use cases. However, training and deploying such gigantic LLMs in resource-constrained, real-world healthcare situations remains challenging. This study addresses the limited support available to visually impaired users and speakers of low-resource languages such as Hindi who require medical assistance in rural environments. We propose PDFTEMRA (Performant Distilled Frequency Transformer Ensemble Model with Random Activations), a compact transformer-based architecture that integrates model distillation, frequency-domain modulation, ensemble learning, and randomized activation patterns to reduce computational cost while preserving language understanding performance. The model is trained and evaluated on medical question-answering and consultation datasets tailored to Hindi and accessibility scenarios, and its performance is compared against standard NLP state-of-the-art model baselines. Results demonstrate that PDFTEMRA achieves comparable performance with substantially lower computational requirements, indicating its suitability for accessible, inclusive, low-resource medical NLP applications.