An Empirical Study of Bitwise Operators Intuitiveness through Performance Metrics

TL;DR

This empirical study evaluates how different groups understand bitwise operators by measuring response times and error rates, revealing that some operators are less intuitive despite similar overall complexity.

Contribution

It provides new insights into the human understanding of bitwise operators across diverse programming backgrounds through experimental performance metrics.

Findings

Operators like OR, NOT, and Left Shift significantly affect response times

No overall increase in task duration for complex operators

Certain operators are less intuitive, indicating potential for redesign

Abstract

Objectives: This study aims to investigate the readability and understandability of bitwise operators in programming, with the main hypothesis that there will be a difference in the performance metrics (response time and error rate) between participants exposed to various bitwise operators related questions and those who are not. Participants: Participants in this human research study include people without programming background, novice programmers, and university students with varying programming experience (from freshmen to PhD level). There were 23 participants in this study. Study Methods: This study uses a within-subjects experimental design to assess how people with diverse programming backgrounds understand and use bitwise operators. Participants complete tasks in a JavaScript program, and their task completion times and task accuracy are recorded for analysis. Findings: The results indicate that operators can be one of the factors predicting response time, showing a small but significant effect (R-squared = 0.032, F(1, 494) = 16.5, p < .001). Additionally, operators such as OR, NOT, and Left Shift showed statistical significance in task completion times compared to other operators. Conclusions: While the complexity of bitwise operators did not generally result in longer task completion times, certain operators were found to be less intuitive, suggesting the need for further investigation and potential redesign for improved understandability.