Know Your Audience: Do LLMs Adapt to Different Age and Education Levels?

TL;DR

This study evaluates how well large language models adapt their responses to different age and education levels, revealing significant variability and highlighting the need for improved audience-specific adaptability in educational contexts.

Contribution

The paper systematically assesses the readability of LLM-generated answers across diverse audiences, exposing current limitations and emphasizing the importance of enhancing their adaptability.

Findings

Large variations in answer readability across LLMs

Current LLMs do not effectively adapt to audience demographics

Readability ranges are limited, impacting educational usefulness

Abstract

Large language models (LLMs) offer a range of new possibilities, including adapting the text to different audiences and their reading needs. But how well do they adapt? We evaluate the readability of answers generated by four state-of-the-art LLMs (commercial and open-source) to science questions when prompted to target different age groups and education levels. To assess the adaptability of LLMs to diverse audiences, we compare the readability scores of the generated responses against the recommended comprehension level of each age and education group. We find large variations in the readability of the answers by different LLMs. Our results suggest LLM answers need to be better adapted to the intended audience demographics to be more comprehensible. They underline the importance of enhancing the adaptability of LLMs in education settings to cater to diverse age and education levels. Overall, current LLMs have set readability ranges and do not adapt well to different audiences, even when prompted. That limits their potential for educational purposes.