Adaptive REST API Testing with Reinforcement Learning

TL;DR

This paper introduces an adaptive REST API testing method using reinforcement learning to improve exploration efficiency, prioritize operations, and handle dynamic schemas, leading to better coverage and fault detection.

Contribution

The paper presents a novel reinforcement learning-based approach for REST API testing that dynamically prioritizes operations and parameters, addressing limitations of existing tools.

Findings

Outperforms state-of-the-art tools in code coverage and fault detection

Achieves higher operation and request coverage with fewer requests

Effective in testing APIs with dynamic and incomplete schemas

Abstract

Modern web services increasingly rely on REST APIs. Effectively testing these APIs is challenging due to the vast search space to be explored, which involves selecting API operations for sequence creation, choosing parameters for each operation from a potentially large set of parameters, and sampling values from the virtually infinite parameter input space. Current testing tools lack efficient exploration mechanisms, treating all operations and parameters equally (i.e., not considering their importance or complexity) and lacking prioritization strategies. Furthermore, these tools struggle when response schemas are absent in the specification or exhibit variants. To address these limitations, we present an adaptive REST API testing technique that incorporates reinforcement learning to prioritize operations and parameters during exploration. Our approach dynamically analyzes request and response data to inform dependent parameters and adopts a sampling-based strategy for efficient processing of dynamic API feedback. We evaluated our technique on ten RESTful services, comparing it against state-of-the-art REST testing tools with respect to code coverage achieved, requests generated, operations covered, and service failures triggered. Additionally, we performed an ablation study on prioritization, dynamic feedback analysis, and sampling to assess their individual effects. Our findings demonstrate that our approach outperforms existing REST API testing tools in terms of effectiveness, efficiency, and fault-finding ability.