TL;DR
This paper investigates how object detectors can hallucinate objects and parts, leading to false positives in visual part verification, and introduces a new dataset for evaluating this issue.
Contribution
It reveals hallucination issues in object detectors for part verification and presents the first dataset, DelftBikes, for systematic evaluation of this problem.
Findings
Object detectors can hallucinate missing objects and parts.
DelftBikes dataset provides detailed annotations for parts and their presence.
Evaluation shows variability in detector performance on part verification.
Abstract
We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
