Utilizing API Response for Test Refinement

TL;DR

This paper introduces a dynamic test refinement method that uses API responses to improve test case realism and coverage, reducing invalid responses and enhancing testing efficiency.

Contribution

It proposes a novel approach leveraging response messages for iterative test refinement, improving test quality without extensive learning requirements.

Findings

Reduced 4xx response rates in testing.

Achieved higher coverage with fewer API requests.

Outperformed existing search-based testing tools.

Abstract

Most of the web services are offered in the form of RESTful APIs. This has led to an active research interest in API testing to ensure the reliability of these services. While most of the testing techniques proposed in the past rely on the API specification to generate the test cases, a major limitation of such an approach is that in the case of an incomplete or inconsistent specification, the test cases may not be realistic in nature and would result in a lot of 4xx response due to invalid input. This is indicative of poor test quality. Learning-based approaches may learn about valid inputs but often require a large number of request-response pairs to learn the constraints, making it infeasible to be readily used in the industry. To address this limitation, this paper proposes a dynamic test refinement approach that leverages the response message. The response is used to infer the point in the API testing flow where a test scenario fix is required. Using an intelligent agent, the approach adds constraints to the API specification that are further used to generate a test scenario accounting for the learned constraint from the response. Following a greedy approach, the iterative learning and refinement of test scenarios are obtained from the API testing system. The proposed approach led to a decrease in the number of 4xx responses, taking a step closer to generating more realistic test cases with high coverage that would aid in functional testing. A high coverage was obtained from a lesser number of API requests, as compared with the state-of-the-art search-based API Testing tools.