Can LLMs replace Neil deGrasse Tyson? Evaluating the Reliability of LLMs as Science Communicators

TL;DR

This study evaluates the reliability of large language models as science communicators using a novel dataset, revealing strengths of certain open-access models and exposing significant limitations in current models' factual accuracy and trustworthiness.

Contribution

Introduces SCiPS-QA, a new dataset for scientific question-answering, and benchmarks multiple LLMs, highlighting their strengths and weaknesses in science communication tasks.

Findings

Llama-3-70B often outperforms GPT-4 Turbo in evaluation.

Most open-access models underperform compared to GPT-4 Turbo.

GPT models frequently fail to reliably verify responses and can deceive human evaluators.

Abstract

Large Language Models (LLMs) and AI assistants driven by these models are experiencing exponential growth in usage among both expert and amateur users. In this work, we focus on evaluating the reliability of current LLMs as science communicators. Unlike existing benchmarks, our approach emphasizes assessing these models on scientific questionanswering tasks that require a nuanced understanding and awareness of answerability. We introduce a novel dataset, SCiPS-QA, comprising 742 Yes/No queries embedded in complex scientific concepts, along with a benchmarking suite that evaluates LLMs for correctness and consistency across various criteria. We benchmark three proprietary LLMs from the OpenAI GPT family and 13 open-access LLMs from the Meta Llama-2, Llama-3, and Mistral families. While most open-access models significantly underperform compared to GPT-4 Turbo, our experiments identify Llama-3-70B as a strong competitor, often surpassing GPT-4 Turbo in various evaluation aspects. We also find that even the GPT models exhibit a general incompetence in reliably verifying LLM responses. Moreover, we observe an alarming trend where human evaluators are deceived by incorrect responses from GPT-4 Turbo.