Fine-Tuning BERT for Domain-Specific Question Answering: Toward Educational NLP Resources at University Scale

TL;DR

This paper demonstrates that fine-tuning BERT on a university-specific question-answer dataset improves its ability to answer educational questions, paving the way for domain-specific educational NLP tools.

Contribution

It introduces a novel fine-tuning approach for BERT using university course data, filling a gap in domain-specific educational NLP resources.

Findings

Fine-tuning BERT improves question-answering accuracy in educational domain

Constructed a dataset of 1,203 university-related QA pairs in SQuAD format

Demonstrated feasibility of adapting foundation models to university course materials

Abstract

Prior work on scientific question answering has largely emphasized chatbot-style systems, with limited exploration of fine-tuning foundation models for domain-specific reasoning. In this study, we developed a chatbot for the University of Limerick's Department of Electronic and Computer Engineering to provide course information to students. A custom dataset of 1,203 question-answer pairs in SQuAD format was constructed using the university book of modules, supplemented with manually and synthetically generated entries. We fine-tuned BERT (Devlin et al., 2019) using PyTorch and evaluated performance with Exact Match and F1 scores. Results show that even modest fine-tuning improves hypothesis framing and knowledge extraction, demonstrating the feasibility of adapting foundation models to educational domains. While domain-specific BERT variants such as BioBERT and SciBERT exist for biomedical and scientific literature, no foundation model has yet been tailored to university course materials. Our work addresses this gap by showing that fine-tuning BERT with academic QA pairs yields effective results, highlighting the potential to scale towards the first domain-specific QA model for universities and enabling autonomous educational knowledge systems.