Difficult for Whom? A Study of Japanese Lexical Complexity

TL;DR

This study investigates Japanese lexical complexity prediction and complex word identification, revealing challenges in personalization and the limited impact of model adaptation, with implications for language processing tasks involving diverse populations.

Contribution

It verifies dataset representativeness, explores personalization effects, and assesses model adaptation challenges in Japanese lexical complexity tasks.

Findings

Group mean models perform similarly to individual models in CWI.

Personalized LCP models are difficult to develop.

Finetuned BERT adaptation yields marginal improvements.

Abstract

The tasks of lexical complexity prediction (LCP) and complex word identification (CWI) commonly presuppose that difficult to understand words are shared by the target population. Meanwhile, personalization methods have also been proposed to adapt models to individual needs. We verify that a recent Japanese LCP dataset is representative of its target population by partially replicating the annotation. By another reannotation we show that native Chinese speakers perceive the complexity differently due to Sino-Japanese vocabulary. To explore the possibilities of personalization, we compare competitive baselines trained on the group mean ratings and individual ratings in terms of performance for an individual. We show that the model trained on a group mean performs similarly to an individual model in the CWI task, while achieving good LCP performance for an individual is difficult. We also experiment with adapting a finetuned BERT model, which results only in marginal improvements across all settings.