Empirical Comparison of Black-box Test Case Generation Tools for RESTful APIs
Davide Corradini, Amedeo Zampieri, Michele Pasqua, Mariano Ceccato

TL;DR
This study empirically compares four black-box test case generation tools for RESTful APIs, evaluating their robustness and coverage on real-world services to guide developers in tool selection.
Contribution
It provides the first comprehensive empirical comparison of existing REST API testing tools, highlighting their strengths and weaknesses in practical scenarios.
Findings
RESTler is the most robust, testing all case studies successfully.
RestTestGen achieves the highest test coverage.
Other tools experienced crashes during testing.
Abstract
In literature, we can find research tools to automatically generate test cases for RESTful APIs, addressing the specificity of this particular programming domain. However, no direct comparison of these tools is available to guide developers in deciding which tool best fits their REST API project. In this paper, we present the results of an empirical comparison of automated black-box test case generation approaches for REST APIs. We surveyed the available black-box testing tools that have been proposed in recent literature, finding four usable prototypes: RestTestGen, RESTler, bBOXRT and RESTest. We used these tools to generate test cases for 14 real-world REST services. Then, testing results have been analyzed and compared in terms of robustness (i.e., success rate) and test coverage. Among the considered tools, RESTler appears to be the most solid, able to successfully test all case…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Software System Performance and Reliability · Software Engineering Techniques and Practices
