Generating Educational Materials with Different Levels of Readability using LLMs

TL;DR

This paper explores the use of large language models to generate educational materials at different readability levels, assessing their effectiveness and challenges in maintaining content accuracy and appropriate difficulty.

Contribution

It introduces the leveled-text generation task and evaluates multiple LLMs' ability to produce content at specified readability levels with minimal supervision.

Findings

Few-shot prompting improves readability control and content preservation.

LLaMA-2 70B better achieves target difficulty levels.

GPT-3.5 better preserves original meaning.

Abstract

This study introduces the leveled-text generation task, aiming to rewrite educational materials to specific readability levels while preserving meaning. We assess the capability of GPT-3.5, LLaMA-2 70B, and Mixtral 8x7B, to generate content at various readability levels through zero-shot and few-shot prompting. Evaluating 100 processed educational materials reveals that few-shot prompting significantly improves performance in readability manipulation and information preservation. LLaMA-2 70B performs better in achieving the desired difficulty range, while GPT-3.5 maintains original meaning. However, manual inspection highlights concerns such as misinformation introduction and inconsistent edit distribution. These findings emphasize the need for further research to ensure the quality of generated educational content.