Evolutionary perspective of large language models on shaping research insights into healthcare disparities

TL;DR

This study investigates how large language models evolve in their ability to identify and analyze research themes in healthcare disparities, demonstrating their potential to aid understanding of scientific trends and impact over time.

Contribution

It introduces a framework for tracking LLMs' thematic evolution in healthcare disparities research, comparing different models' capabilities and relevance to scientific impact.

Findings

LLMs' outputs correlate with actual research impact and trends.

Different models vary in depth and breadth of theme identification.

The approach helps navigate evolving research landscapes.

Abstract

Introduction. Advances in large language models (LLMs) offer a chance to act as scientific assistants, helping people grasp complex research areas. This study examines how LLMs evolve in healthcare disparities research, with attention to public access to relevant information. Methods. We studied three well-known LLMs: ChatGPT, Copilot, and Gemini. Each week, we asked them a consistent prompt about research themes in healthcare disparities and tracked how their answers changed over a one-month period. Analysis. The themes produced by the LLMs were categorized and cross-checked against H-index values from the Web of Science to verify relevance. This dual approach shows how the outputs of LLMs develop over time and how such progress could help researchers navigate trends. Results. The outputs aligned with actual scientific impact and trends in the field, indicating that LLMs can help people understand the healthcare disparities landscape. Time-series comparisons showed differences among the models in how broadly and deeply they identified and classified themes. Conclusion. The study offers a framework that uses the evolution of multiple LLMs to illuminate AI tools for studying healthcare disparities, informing future research and public engagement strategies.