From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Science

TL;DR

This study demonstrates that AI-generated simplified summaries of scientific articles improve public understanding and perceptions of scientists, suggesting AI's role in making science more accessible and trustworthy.

Contribution

The paper introduces a method using GPT-4 to generate lay summaries that significantly enhance comprehension and perception of science compared to traditional summaries.

Findings

GPT-4 summaries doubled effect size of simplicity.

Participants understood scientific content better with GPT summaries.

GPT summaries made scientists seem more credible and trustworthy.

Abstract

This paper evaluated the effectiveness of using generative AI to simplify science communication and enhance the public's understanding of science. By comparing lay summaries of journal articles from PNAS, yoked to those generated by AI, this work first assessed linguistic simplicity differences across such summaries and public perceptions in follow-up experiments. Specifically, Study 1a analyzed simplicity features of PNAS abstracts (scientific summaries) and significance statements (lay summaries), observing that lay summaries were indeed linguistically simpler, but effect size differences were small. Study 1b used a large language model, GPT-4, to create significance statements based on paper abstracts and this more than doubled the average effect size without fine-tuning. Study 2 experimentally demonstrated that simply-written GPT summaries facilitated more favorable perceptions of scientists (they were perceived as more credible and trustworthy, but less intelligent) than more complexly-written human PNAS summaries. Crucially, Study 3 experimentally demonstrated that participants comprehended scientific writing better after reading simple GPT summaries compared to complex PNAS summaries. In their own words, participants also summarized scientific papers in a more detailed and concrete manner after reading GPT summaries compared to PNAS summaries of the same article. AI has the potential to engage scientific communities and the public via a simple language heuristic, advocating for its integration into scientific dissemination for a more informed society.