ELEVATE-GenAI: Reporting Guidelines for the Use of Large Language Models in Health Economics and Outcomes Research: an ISPOR Working Group on Generative AI Report

TL;DR

This paper introduces the ELEVATE GenAI framework and checklist, providing structured reporting guidelines for health economics research involving large language models, aiming to improve transparency, accuracy, and reproducibility.

Contribution

It presents the first comprehensive reporting framework and checklist tailored for LLM use in HEOR studies, developed through literature review and expert input.

Findings

Framework covers 10 key domains including bias and reproducibility.

Checklist demonstrated usability in two diverse HEOR case studies.

Framework addresses a critical gap in standardized reporting for LLM-assisted research.

Abstract

Introduction: Generative artificial intelligence (AI), particularly large language models (LLMs), holds significant promise for Health Economics and Outcomes Research (HEOR). However, standardized reporting guidance for LLM-assisted research is lacking. This article introduces the ELEVATE GenAI framework and checklist - reporting guidelines specifically designed for HEOR studies involving LLMs. Methods: The framework was developed through a targeted literature review of existing reporting guidelines, AI evaluation frameworks, and expert input from the ISPOR Working Group on Generative AI. It comprises ten domains, including model characteristics, accuracy, reproducibility, and fairness and bias. The accompanying checklist translates the framework into actionable reporting items. To illustrate its use, the framework was applied to two published HEOR studies: one focused on systematic literature review tasks and the other on economic modeling. Results: The ELEVATE GenAI framework offers a comprehensive structure for reporting LLM-assisted HEOR research, while the checklist facilitates practical implementation. Its application to the two case studies demonstrates its relevance and usability across different HEOR contexts. Limitations: Although the framework provides robust reporting guidance, further empirical testing is needed to assess its validity, completeness, usability, as well as its generalizability across diverse HEOR use cases. Conclusion: The ELEVATE GenAI framework and checklist address a critical gap by offering structured guidance for transparent, accurate, and reproducible reporting of LLM-assisted HEOR research. Future work will focus on extensive testing and validation to support broader adoption and refinement.