Survey on Reasoning Capabilities and Accessibility of Large Language Models Using Biology-related Questions

TL;DR

This survey evaluates the reasoning capabilities and accessibility of large language models in biomedicine by testing them with biology-related questions, highlighting recent improvements and user impact.

Contribution

It introduces a new set of biology-related questions to assess reasoning in top LLMs and extends previous research on biological literature retrieval.

Findings

Improved reasoning abilities in top LLMs over time

Enhanced depth in biological literature retrieval

Quantified user-perceived improvements

Abstract

This research paper discusses the advances made in the past decade in biomedicine and Large Language Models. To understand how the advances have been made hand-in-hand with one another, the paper also discusses the integration of Natural Language Processing techniques and tools into biomedicine. Finally, the goal of this paper is to expand on a survey conducted last year (2023) by introducing a new list of questions and prompts for the top two language models. Through this survey, this paper seeks to quantify the improvement made in the reasoning abilities in LLMs and to what extent those improvements are felt by the average user. Additionally, this paper seeks to extend research on retrieval of biological literature by prompting the LLM to answer open-ended questions in great depth.