Investigating the Impact of Vocabulary Difficulty and Code Naturalness on Program Comprehension

TL;DR

This study investigates how vocabulary difficulty and code naturalness influence program comprehension, aiming to improve readability prediction by analyzing correlations with source code characteristics.

Contribution

It introduces a novel approach to assess code readability by examining vocabulary difficulty and naturalness, and explores their potential to enhance prediction models.

Findings

Code naturalness correlates with readability scores.

Vocabulary difficulty impacts understandability assessments.

Naturalness and vocabulary metrics can improve prediction accuracy.

Abstract

Context: Developers spend most of their time comprehending source code during software development. Automatically assessing how readable and understandable source code is can provide various benefits in different tasks, such as task triaging and code reviews. While several studies have proposed approaches to predict software readability and understandability, most of them only focus on local characteristics of source code. Besides, the performance of understandability prediction is far from satisfactory. Objective: In this study, we aim to assess readability and understandability from the perspective of language acquisition. More specifically, we would like to investigate whether code readability and understandability are correlated with the naturalness and vocabulary difficulty of source code. Method: To assess code naturalness, we adopted the cross-entropy metric, while we use a manually crafted list of code elements with their assigned advancement levels to assess the vocabulary difficulty. We will conduct a statistical analysis to understand their correlations and analyze whether code naturalness and vocabulary difficulty can be used to improve the performance of code readability and understandability prediction methods. The study will be conducted on existing datasets.