SolarGPT-QA: A Domain-Adaptive Large Language Model for Educational Question Answering in Space Weather and Heliophysics

TL;DR

SolarGPT-QA is a specialized large language model designed for educational question answering in space weather and heliophysics, trained on scientific literature and optimized for clarity and pedagogical effectiveness.

Contribution

It introduces a domain-adaptive LLM built on LLaMA-3, trained with scientific literature and question-answer data, and evaluated with a novel LLM-as-judge framework.

Findings

Outperforms general models in zero-shot space weather questions.

Achieves competitive results with instruction-tuned models for educational explanations.

Combining domain pretraining and fine-tuning improves scientific accuracy and pedagogy.

Abstract

Solar activity, including solar flares, coronal mass ejections (CMEs), and geomagnetic storms can significantly impact satellites, aviation, power grids, data centers, and space missions. Extreme solar events can cause substantial economic damage with limited advance warning, underscoring the importance of early warning systems, accurate forecasting, and effective education in space science. Although large language models (LLMs) perform well on general tasks, they often lack domain specific knowledge and pedagogical capability to clearly explain complex space science concepts. We introduce SolarGPT-QA, a question answering system based on a domain adapted large language model built on the LLaMA-3 base model. The model is trained using scientific literature and large scale question and answer data generated with GPT-4 and refined using Grok-3 in a student friendly storytelling style. To evaluate response quality, we employ an LLM-as-judge evaluation framework, where a strong reference model assesses generated answers using structured criteria including scientific accuracy, clarity, completeness, and pedagogical effectiveness. Results show that SolarGPT-QA performs strongly relative to general purpose models in zero shot settings and achieves competitive performance compared to instruction tuned models for educational explanations in space weather and heliophysics. Ablation studies indicate that combining domain adaptive pretraining with fine tuning is important for balancing scientific accuracy and educational effectiveness.