# Benchmarking Web API Quality -- Revisited

## TL;DR

This study revisits a 3-month web API benchmarking effort from 2015, repeating it in 2018, and compares availability, latency, and security preferences to understand quality variations over time and geography.

## Contribution

It provides a comparative analysis of web API quality over time, introduces new security preference assessments, and discusses implications for API reliability and developer tools.

## Key findings

- API quality varies by geo-distribution of clients.
- API availability and latency fluctuate over time and experiments.
- Security preferences of providers differ and impact API trustworthiness.

## Abstract

Modern applications increasingly interact with web APIs -- reusable components, deployed and operated outside the application, and accessed over the network. Their existence, arguably, spurs application innovations, making it easy to integrate data or functionalities. While previous work has analyzed the ecosystem of web APIs and their design, little is known about web API quality at runtime. This gap is critical, as qualities including availability, latency, or provider security preferences can severely impact applications and user experience. In this paper, we revisit a 3-month, geo-distributed benchmark of popular web APIs, originally performed in 2015. We repeat this benchmark in 2018 and compare results from these two benchmarks regarding availability and latency. We furthermore introduce new results from assessing provider security preferences, collected both in 2015 and 2018, and results from our attempts to reach out to API providers with the results from our 2015 experiments. Our extensive experiments show that web API qualities vary 1.) based on the geo-distribution of clients, 2.) during our individual experiments, and 3.) between the two experiments. Our findings provide evidence to foster the discussion around web API quality, and can act as a basis for the creation of tools and approaches to mitigate quality issues.