QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIs

TL;DR

QuickREST introduces an automated, property-based testing approach for RESTful APIs described by OpenAPI, enabling low-effort fault detection and validation that adapts to API evolution.

Contribution

This paper presents a novel method for generating property-based tests from OpenAPI documents, facilitating automatic testing and validation of REST APIs.

Findings

Effective fault detection on industrial and open source APIs

Supports automatic evolution of tests with API changes

Helps identify specification-implementation misalignments

Abstract

RESTful APIs are an increasingly common way to expose software systems functionality and it is therefore of high interest to find methods to automatically test and verify such APIs. To lower the barrier for industry adoption, such methods needs to be straightforward to use with a low effort. This paper introduces a method to explore the behaviour of a RESTful API. This is done by using automatic property-based tests produced from OpenAPI documents that describe the REST API under test. We describe how this method creates artifacts that can be leveraged both as property-based test generators and as a source of validation for results (i.e., as test oracles). Experimental results, on both industrial and open source services, indicate how this approach is a low effort way of finding real faults. Furthermore, it supports building additional knowledge about the system under test by automatically exposing misalignment of specification and implementation. Since the tests are generated from the OpenAPI document this method automatically evolves test cases as the REST API evolves.