From Knowledge to Inference: Scaling Laws of Specialized Reasoning on GlobalHealthAtlas

TL;DR

This paper introduces GlobalHealthAtlas, a large multilingual dataset for public health reasoning, along with an LLM-assisted construction pipeline and a domain-specific evaluator to advance safety-critical health AI applications.

Contribution

It presents a new large-scale, multilingual public health dataset, an LLM-assisted data creation process, and a specialized evaluator for assessing reasoning quality in health AI models.

Findings

Dataset covers 15 health domains and 17 languages.

Proposed pipeline improves data quality and consistency.

Evaluator assesses models on accuracy, reasoning, and insightfulness.

Abstract

Public health reasoning requires population level inference grounded in scientific evidence, expert consensus, and safety constraints. However, it remains underexplored as a structured machine learning problem with limited supervised signals and benchmarks. We introduce \textbf{GlobalHealthAtlas}, a large scale multilingual dataset of 280,210 instances spanning 15 public health domains and 17 languages, stratified into three difficulty levels from health literacy to epidemiological and policy reasoning. Instances are derived from openly available public health sources and labeled by language, domain, and difficulty to support supervised learning and slice based evaluation. We further propose large language model (LLM) assisted construction and quality control pipeline with retrieval, duplication, evidence grounding checks, and label validation to improve consistency at scale. Finally, we present a domain aligned evaluator distilled from high confidence judgments of diverse LLMs to assess outputs along six dimensions: Accuracy, Reasoning, Completeness, Consensus Alignment, Terminology Norms, and Insightfulness. Together, these contributions enable reproducible training and evaluation of LLMs for safety critical public health reasoning beyond conventional QA benchmarks.