Requirements of API Documentation: A Case Study into Computer Vision Services

TL;DR

This paper investigates the essential requirements for API documentation in computer vision services, highlighting gaps between industry practices and research, and offers recommendations to improve documentation quality for developers.

Contribution

It extends prior work by surveying developers on valuable documentation artefacts and analyzing real-world documentation in computer vision APIs to identify gaps and best practices.

Findings

Developers value specific API artefacts for understanding non-deterministic behaviour.

Significant gaps exist between industry documentation practices and research recommendations.

Recommendations are provided to improve API documentation for AI-powered services.

Abstract

Using cloud-based computer vision services is gaining traction, where developers access AI-powered components through familiar RESTful APIs, not needing to orchestrate large training and inference infrastructures or curate/label training datasets. However, while these APIs seem familiar to use, their non-deterministic run-time behaviour and evolution is not adequately communicated to developers. Therefore, improving these services' API documentation is paramount-more extensive documentation facilitates the development process of intelligent software. In a prior study, we extracted 34 API documentation artefacts from 21 seminal works, devising a taxonomy of five key requirements to produce quality API documentation. We extend this study in two ways. Firstly, by surveying 104 developers of varying experience to understand what API documentation artefacts are of most value to practitioners. Secondly, identifying which of these highly-valued artefacts are or are not well-documented through a case study in the emerging computer vision service domain. We identify: (i) several gaps in the software engineering literature, where aspects of API documentation understanding is/is not extensively investigated; and (ii) where industry vendors (in contrast) document artefacts to better serve their end-developers. We provide a set of recommendations to enhance intelligent software documentation for both vendors and the wider research community.