Personalization of Code Readability Evaluation Based on LLM Using Collaborative Filtering

TL;DR

This paper introduces a personalized code readability evaluation method using large language models calibrated with collaborative filtering, improving accuracy by accounting for individual developer preferences.

Contribution

It presents a novel approach combining LLMs with collaborative filtering to personalize code readability assessments, addressing variability among developers.

Findings

Enhanced evaluation accuracy demonstrated

Effective calibration of LLM-based assessments

Potential for improved software maintenance insights

Abstract

Code readability is an important indicator of software maintenance as it can significantly impact maintenance efforts. Recently, LLM (large language models) have been utilized for code readability evaluation. However, readability evaluation differs among developers, so personalization of the evaluation by LLM is needed. This study proposes a method which calibrates the evaluation, using collaborative filtering. Our preliminary analysis suggested that the method effectively enhances the accuracy of the readability evaluation using LLMs.