Do RESTful API Design Rules Have an Impact on the Understandability of Web APIs? A Web-Based Experiment with API Descriptions

TL;DR

This study empirically investigates how adherence to RESTful API design rules affects the understandability of Web APIs, demonstrating that rule violations generally decrease comprehension and increase perceived difficulty.

Contribution

It provides the first empirical evidence on the impact of RESTful design rules on API understandability through a controlled web-based experiment.

Findings

Violations of 11 out of 12 rules significantly reduce comprehension.

Most violations are perceived as more difficult to understand.

Demographic attributes do not influence comprehension of rule violations.

Abstract

Context: Web APIs are one of the most used ways to expose application functionality on the Web, and their understandability is important for efficiently using the provided resources. While many API design rules exist, empirical evidence for the effectiveness of most rules is lacking. Objective: We therefore wanted to study 1) the impact of RESTful API design rules on understandability, 2) if rule violations are also perceived as more difficult to understand, and 3) if demographic attributes like REST-related experience have an influence on this. Method: We conducted a controlled Web-based experiment with 105 participants, from both industry and academia and with different levels of experience. Based on a hybrid between a crossover and a between-subjects design, we studied 12 design rules using API snippets in two complementary versions: one that adhered to a "rule" and one that was a "violation" of this rule. Participants answered comprehension questions and rated the perceived difficulty. Results: For 11 of the 12 rules, we found that "violation" performed significantly worse than "rule" for the comprehension tasks. Regarding the subjective ratings, we found significant differences for 9 of the 12 rules, meaning that most violations were subjectively rated as more difficult to understand. Demographics played no role in the comprehension performance for "violation". Conclusions: Our results provide first empirical evidence for the importance of following design rules to improve the understandability of Web APIs, which is important for researchers, practitioners, and educators.