Assessing Python Style Guides: An Eye-Tracking Study with Novice Developers

TL;DR

This eye-tracking study with novice Python developers evaluates how adherence to PEP8 style guide recommendations affects code readability and visual effort, highlighting specific guidelines that improve or hinder understanding.

Contribution

It provides empirical evidence on the impact of PEP8 style guidelines on novice developers' reading effort using eye-tracking metrics.

Findings

Not following the Line Break after an Operator increases eye regressions by 70%.

Most subjects preferred PEP8-compliant code despite some guidelines negatively affecting eye metrics.

Guidelines like True Comparison negatively impacted eye metrics, yet were preferred by subjects.

Abstract

The incorporation and adaptation of style guides play an essential role in software development, influencing code formatting, naming conventions, and structure to enhance readability and simplify maintenance. However, many of these guides often lack empirical studies to validate their recommendations. Previous studies have examined the impact of code styles on developer performance, concluding that some styles have a negative impact on code readability. However, there is a need for more studies that assess other perspectives and the combination of these perspectives on a common basis through experiments. This study aimed to investigate, through eye-tracking, the impact of guidelines in style guides, with a special focus on the PEP8 guide in Python, recognized for its best practices. We conducted a controlled experiment with 32 Python novices, measuring time, the number of attempts, and visual effort through eye-tracking, using fixation duration, fixation count, and regression count for four PEP8 recommendations. Additionally, we conducted interviews to explore the subjects' difficulties and preferences with the programs. The results highlighted that not following the PEP8 Line Break after an Operator guideline increased the eye regression count by 70% in the code snippet where the standard should have been applied. Most subjects preferred the version that adhered to the PEP8 guideline, and some found the left-aligned organization of operators easier to understand. The other evaluated guidelines revealed other interesting nuances, such as the True Comparison, which negatively impacted eye metrics for the PEP8 standard, although subjects preferred the PEP8 suggestion. We recommend practitioners selecting guidelines supported by experimental evaluations.