Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper)
Fitash Ul Haq, Donghwan Shin, Lionel C. Briand, Thomas Stifter, Jun, Wang

TL;DR
This paper introduces an automated method using many-objective search to generate test data for key-points detection DNNs, significantly improving detection of severe mispredictions in facial key-point detection for automotive applications.
Contribution
It presents a novel approach employing many-objective search for automatic test suite generation targeting KP-DNNs, with empirical evaluation and analysis of conditions causing mispredictions.
Findings
Generated test suites can cause over 93% severe mispredictions.
Compared to random search, our method significantly increases misprediction rates.
Identified image conditions correlate with higher misprediction risks.
Abstract
Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time -- where each individual key-point may be critical in the targeted application -- and images can vary a great deal according to many factors. In this paper, we present an approach to automatically generate test data for KP-DNNs using many-objective search. In our experiments, focused on facial key-points detection DNNs developed for an industrial automotive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
