Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries

TL;DR

This paper presents a novel approach to automatically generate simple, learner-friendly dictionary definitions using large language models, along with a new evaluation method and a Japanese dataset.

Contribution

It introduces a reliable LLM-based evaluation framework and a method for iterative simplification to generate learner's dictionary definitions.

Findings

The evaluation method aligns well with human judgments.

Generated definitions are simple and meet the criteria for learner's dictionaries.

The approach achieves high scores on the proposed evaluation metrics.

Abstract

We study dictionary definition generation (DDG), i.e., the generation of non-contextualized definitions for given headwords. Dictionary definitions are an essential resource for learning word senses, but manually creating them is costly, which motivates us to automate the process. Specifically, we address learner's dictionary definition generation (LDDG), where definitions should consist of simple words. First, we introduce a reliable evaluation approach for DDG, based on our new evaluation criteria and powered by an LLM-as-a-judge. To provide reference definitions for the evaluation, we also construct a Japanese dataset in collaboration with a professional lexicographer. Validation results demonstrate that our evaluation approach agrees reasonably well with human annotators. Second, we propose an LDDG approach via iterative simplification with an LLM. Experimental results indicate that definitions generated by our approach achieve high scores on our criteria while maintaining lexical simplicity.