Enhancing Health Fact-Checking with LLM-Generated Synthetic Data

TL;DR

This paper introduces a synthetic data generation pipeline using large language models to improve health-related fact-checking accuracy, demonstrating significant performance gains on public datasets.

Contribution

The study presents a novel LLM-based synthetic data augmentation method specifically designed for health fact-checking, enhancing model performance with limited annotated data.

Findings

F1 scores increased by up to 0.019 on PubHealth

F1 scores increased by up to 0.049 on SciFact

Synthetic data improves health fact-checking models

Abstract

Fact-checking for health-related content is challenging due to the limited availability of annotated training data. In this study, we propose a synthetic data generation pipeline that leverages large language models (LLMs) to augment training data for health-related fact checking. In this pipeline, we summarize source documents, decompose the summaries into atomic facts, and use an LLM to construct sentence-fact entailment tables. From the entailment relations in the table, we further generate synthetic text-claim pairs with binary veracity labels. These synthetic data are then combined with the original data to fine-tune a BERT-based fact-checking model. Evaluation on two public datasets, PubHealth and SciFact, shows that our pipeline improved F1 scores by up to 0.019 and 0.049, respectively, compared to models trained only on the original data. These results highlight the effectiveness of LLM-driven synthetic data augmentation in enhancing the performance of health-related fact-checkers.

Tables

Table 1. Table 1 : Statistics of the data.

Data	Training			Test			Document		Claim
	Total	True	False	Total	True	False	Avg sent.	Avg word	Avg sent.	Avg word
PubHealth	5,863	4,101	1,762	987	599	388	20.42	511.26	1.05	12.54
SciFact	957	616	341	338	216	122	10.15	245.83	1.00	12.19

Table 2. Table 2 : Results on PubHealth and SciFact.

Synthetic proportion	PubHealth									SciFact
	500			1,000			1,500
	P	R	F	P	R	F	P	R	F	P	R	F
0% (baseline)	0.949	0.656	0.776	0.960	0.674	0.792	0.859	0.728	0.812	0.716	0.769	0.741
10%	0.964	0.626	0.759	0.963	0.658	0.782	0.891	0.780	0.831	0.718	0.712	0.714
20%	0.903	0.621	0.736	0.849	0.768	0.806	0.912	0.741	0.813	0.776	0.686	0.727
30%	0.842	0.723	0.778	0.904	0.708	0.794	0.902	0.746	0.815	0.741	0.738	0.737
40%	0.842	0.728	0.781	0.869	0.711	0.782	0.911	0.732	0.811	0.745	0.759	0.752
50%	0.741	0.780	0.760	0.838	0.733	0.782	0.892	0.744	0.811	0.687	0.832	0.751
60%	0.812	0.765	0.788	0.938	0.628	0.752	0.888	0.743	0.808	0.672	0.925	0.775
70%	0.686	0.838	0.754	0.950	0.664	0.782	0.882	0.771	0.822	0.658	0.985	0.789
80%	0.728	0.798	0.761	0.909	0.664	0.767	0.855	0.770	0.810	0.641	0.984	0.777
90%	0.843	0.746	0.792	0.896	0.720	0.798	0.833	0.796	0.813	0.648	0.986	0.782
100%	0.664	0.917	0.770	0.948	0.666	0.782	0.810	0.825	0.816	0.666	0.979	0.792

Full text

1 Introduction

Fact-checking is the process of evaluating the truthfulness of claims by systematically comparing them against credible and authoritative sources of evidence [1, 2]. This process is crucial to combating misinformation, especially in public health, where false or misleading medical information can directly harm public well being [3]. Ensuring that health‐related claims are accurate is not only fundamental to upholding scientific integrity, but also essential for preserving public trust.

One key challenge in developing reliable fact-checkers in medicine is the lack of labeled training data. Fact-checking in the health domain requires specialized medical expertise, making the annotation process costly and time-consuming [4]. As a result, models trained on general-purpose fact-checking corpora often struggle to generalize to medical claims, highlighting the critical need for more domain-specific data. However, existing datasets have notable limitations when applied to health-related contexts. For instance, the widely used FEVER dataset contains over 185,000 synthetic claims derived from Wikipedia, but it does not require domain knowledge and therefore lacks applicability to medical fact-checking tasks [5]. While PubHealth offers approximately 11,000 journalist-verified claims focused on public health, its topic coverage is narrow and biased toward false statements [1]. Similarly, SciFact is tailored to verifying scientific claims against paper abstracts but comprises only about 1,400 annotated instances—an insufficient quantity for training large language models [6].

In this study, we introduce a synthetic data generation pipeline for fact‐checking. We show that, by leveraging synthetic data, it is possible to fine-tune a smaller model (e.g., BERT) to effectively verify multiple facts within grounding documents. Specifically, the data synthesis pipeline first summarizes each document and extracts its atomic facts while simultaneously splitting the original text into individual sentences. A large language model (LLM) is then used to construct a sentence–fact support table, marking whether each sentence supports each fact. To generate synthetic data, we sample a subset of sentences, select one of the extracted facts as a synthetic claim, and automatically assign its veracity label by consulting the support table. Finally, we merge these synthetic examples with the original data to fine-tune a BERT-based fact-checking model, thus creating a richer training set. Specifically, our approach led to F1 score improvements of up to 0.019 on PubHealth and 0.049 on SciFact compared to models trained only on the original data.

Our contributions are as follows: (1) We propose an LLM-driven pipeline that constructs claim–text pairs from arbitrary documents using atomic facts, (2) We implement a BERT-based FactChecker using the synthetic data, (3) Evaluation on the PubHealth and SciFact datasets shows that, by leveraging synthetic data, it is possible to fine-tune a smaller model (e.g., BERT) to effectively verify multiple facts within grounding documents.

2 Methodology

2.1 Overview

In this study, we want to build a model, FactChecker, to decide whether each claim $c$ can be supported (true) or unsupported (false), according to a document $D$. To augment an existing dataset, we propose to use the original $D$ in the training data to construct a dataset of the text paired with claims with binary labels. In the synthetic data, the text is composed of sentences randomly selected from $D$; the claim corresponds to an atomic fact, and the label indicates whether the atomic fact is supported by the synthetic text.

Therefore, our proposed pipeline consists of 4 key steps (Figure 1): (1) decomposing the original document to generate summaries and extract atomic facts, (2) constructing a sentence-fact table by mapping atomic facts to segmented sentences from the document, (3) generating synthetic data using the sentence-fact table, and (4) leveraging BERT-based models to train classifiers and make predictions.

2.2 Generating synthetic data

Document summarization

Our pipeline begins with a text summarization step. The primary goal of this stage is to generate a concise and coherent summary of each original document, which serves several purposes. First, summarization reduces the likelihood that a single fact is distributed across multiple sentences, thereby facilitating more precise and isolated extraction of atomic facts in subsequent steps. Second, by generating summaries distinct from the original documents, we introduce greater textual diversity, which can enhance the robustness and generalizability of downstream fact extraction and verification processes. This step ensures that the information is both condensed and reorganized, providing a clear foundation for the accurate identification of discrete facts.

Here, we used the GPT-4 Turbo model (model version turbo-2024-04-09, API version 2024-12-01-preview) to generate the summary using the prompt in Figure 2A. We assume that these LLM-generated summaries are factually consistent with the original document [7].

Fact decomposition

We then use the GPT-4 to decompose each sentence in the summary into a series of atomic facts. Following the definition in [8, 7], an atomic fact is characterized as the most basic, indivisible declarative sentence, each conveying one piece of information.

We design prompts that use concrete fact-decomposition examples to instruct the model to identify and extract detailed information and present it in a structured, orderly format (Figure 2). This approach ensured that the resulting atomic facts were both precise and comprehensive, facilitating more reliable downstream processing.

Sentence-fact table construction

After extracting individual atomic facts from the original documents, we organized the information into a structured sentence-fact table for each document. In this table, each row represents a sentence from the document, while each column corresponds to an atomic fact identified from the extraction process.

We then performed an entailment check to verify the relationship between each sentence and each atomic fact, and populated the table with indicators (true or false) of whether each sentence entails a particular atomic fact. Notably, multiple sentences could entail the same fact. One sentence could entail multiple facts, too.

Proportional sampling synthesis

Next, we generated synthetic data using the sentence-fact tables. Specifically, $p\%$ sentences were randomly selected from each original document (referred to as “Synthetic proportion”) and concatenated to form a new piece of text. A corresponding atomic fact was then chosen from the sentence-fact table to serve as a claim. The label for each synthetic instance was determined by referencing the sentence-fact table: if at least one of the selected sentences supported the chosen fact, the instance was labeled as “true”; otherwise, it was labeled as “false”.

2.3 FactChecker development

We adopted a BERT-based “pretrained encoder + classification head” architecture as our encoder [9]. The two input texts (a claim and its corresponding document) were concatenated into a single sequence formatted as “[CLS] claim [SEP] document [SEP]”. This sequence was then fed through a BERT model, and the final‐layer hidden state of the [CLS]was used as the semantic representation of the pair. We then applied dropout before passing it through a single linear feedforward layer that output two logits. A softmax function was used to compute the probability. All parameters—including both the BERT encoder and the classification head were fine-tuned jointly by minimizing the cross-entropy loss on our binary labels.

3 Experimental Settings

Datasets

To evaluate the performance of our overall model, we conducted experiments on two benchmark datasets: PubHealth and SciFact (Table 1).

PubHealth contains 9,832 document-claim pairs and their corresponding manually verified labels [1]. The labels contain four values: true, false, mixture, and unproven. Considering that our task mainly focused on binary classification, only true and false labeled instances were included. To facilitate effective document decomposition, we also excluded documents that were either too short ($\leq 3$ sentences) or too long ($\geq 40$ sentences). Finally, we obtained a processed set of 5,863 instances for training and 987 instances for validation. In our subsequent experiments, we incrementally, randomly selected 500, 1,000, and 1,500 instances from the training dataset to generate synthetic data. To ensure the label balance, each subset comprised an equal number of instances from both labels.

SciFact is a collection of 1,409 claims verified against 5,183 abstracts [10]. For each claim, the corresponding abstracts were annotated as SUPPORT, CONTRADICT, or NEUTRAL with rationale sentences (also referred to as “evidence”) from the abstracts. Here, we only picked the pairs with the label of SUPPORT (true) and CONTRADICT (false). Since the ground truth for the test set was not published, we used the standard training set (957 pairs) for training and the development set (338 pairs) for testing.

Evaluation Metrics

The precision, recall, and F1 scores were calculated.

Model settings

All experiments were conducted on a Red Hat Enterprise Linux 8.10 (64-bit) system with dual Intel Xeon Gold 6226R CPUs (64 logical cores, 2.90 GHz) and two NVIDIA RTX A6000 GPUs, each with 48 GB of VRAM. In our experiments, we fine-tuned the AllenAI SciBERT model (allenai/scibert_scivocab_uncased) using Hugging Face’s Transformers library. All experiments were conducted using Python 3.11.11, PyTorch 2.5.1+cu121 (CUDA 12.1), Hugging Face Transformers 4.50.3, and Tokenizers 0.21.1. We fine-tuned the model for 10 epochs with a learning rate of $2\times 10^{-5}$. The batch size was 16, and the AdamW optimizer with a weight decay of $1\times 10^{-2}$ is employed.

4 Results and Discussion

4.1 Results on PubHealth and SciFact

Table 2 shows the results on the PubHealth dataset, where column “synthetic proportion” suggests the proportion of sentences selected from the original documents (Section 2.2). We combined the synthetic data with the original training set to form an augmented training set. Note that when the proportion is 0%, no synthetic data was incorporated. In this case, the model was trained solely on the original dataset, which serves as the baseline for comparison. We then compared the model’s performance when trained with the augmented dataset versus using only the original training data. Table 2 shows that the best proportion in each subset all achieves higher performance compared with only using the original training data.

On the subset with 500 instances, the best-performing system was trained using 90% of sentences from the original documents to construct the synthetic data, achieving an F1 score of 0.792. Out of 10 experimental settings, four outperformed the baseline. These results indicate that the effectiveness of synthesized data in enhancing the FactChecker ’s performance depends on selecting an appropriate proportion that provides meaningful and informative training examples.

A similar trend is observed across larger subsets. On the subset with 1,000 instances, employing just 20% of the original sentences led to an impressive F1 score of 0.806, notably higher than the baseline of 0.792. Here, 3 out of the 10 experimental settings overperformed the baseline. The results are even more compelling on the 1,500-instance subset: selecting only 10% of sentences yielded the highest F1 score of 0.831, compared to the baseline of 0.812. In this case, a remarkable 6 out of 10 settings surpassed the baseline, underscoring the robust potential of fine-tuned proportions for maximizing system performance.

Table 2 also shows the results on the SciFact dataset. For most proportions of synthetic data, models’ performance surpassed that of those fine-tuned with only the original data. Notably, the highest F1 score was observed when the proportion was set to 100%, reaching 0.792 – substantially higher than the 0.741 achieved using only the original dataset.

4.2 A pilot study in detecting hallucinations

To demonstrate the effectiveness of FactChecker, we conducted a pilot study focused on detecting hallucinations in LLM-generated text summaries. For each summary, we constructed a sentence-fact table using both the original document and the corresponding LLM-generated summary, following a similar approach as described in Section 2.2. Our FactChecker was then applied to populate the table with values (true or false), indicating whether each sentence supported a given decomposed fact. If all the values in a column were 0, we marked that column as abnormal, suggesting that the corresponding fact may not be supported by any sentence in the original document. Such cases likely indicate hallucinations produced by LLM. Finally, we verified these cases manually.

In this pilot study, we examined 500 instances from the PubHealth dataset that were not included in our earlier experiments. We used our BERT-based FactChecker trained on the 1,500 subsets to assign values in the sentence-fact tables for these new cases. This process revealed two abnormal instances. In the first case (Figure 3), the abnormality was caused by hallucination, in which LLM appeared to use external knowledge or fabricate facts not present in the source. In the second case (Figure 4), the abnormal fact identified in the summary could only be inferred by combining information from several sentences, but was not directly supported by any single sentence. The first scenario is an ideal demonstration for our pilot study, whereas the second one highlights a potential future research direction.

4.3 Discussion

Based on the results obtained from Table 2, several general patterns can be identified. First, we found that increasing the amount of data allows the FactChecker to capture more knowledge and, consequently, achieve better performance. The best results were achieved with F1 scores of 0.792, 0.806, and 0.831 for the 500, 1,000, and 1,500 subsets, respectively. Second, the optimal proportion of sentences selected from the original documents varies across subsets, indicating that there is no universally optimal selection strategy. Thus, further investigations are needed to establish a general model that consistently uses a fixed proportion for all cases.

While we only experimented with several settings in this study, it is worth noting that our proposed data synthesis procedure can be repeated multiple times to efficiently generate a large volume of synthetic data. Furthermore, adjusting the proportion of sentences retained from the original documents could further enhance the flexibility and diversity of this synthetic data generation pipeline.

This study has several limitations. First, our study mainly focused on the two datasets, PubHealth and SciFact, which constrain the generalizability of this study. Second, in our experiment, a single BERT-based FactChecker is employed. In our future work, more BERT models can be tried to discover more innovative results.

5 Conclusion

In this study, we proposed an LLM-driven synthetic data generation pipeline. The sentence-fact table generated through this pipeline, along with the resulting synthetic data, brings new insights into fact-checking tasks. Firstly, the synthetic data intuitively helps solve the insufficiency in training data. Meanwhile, the experiments on the two benchmark datasets with our FactChecker also provide meaningful observations. Based on this pipeline, we further show that it is possible to fine-tune a smaller model to effectively verify facts within grounding documents and detect hallucinations in LLM-generated text. In conclusion, the novelty proposed in this study provides a potential feasible solution and future development direction for the field of fact-checking.

Acknowledgment: This project was sponsored by the National Library of Medicine grants R01LM014344, R01LM014306, and R01LM014573.