Benchmarking Off-The-Shelf Solutions to Robotic Assembly Tasks
Wenzhao Lian, Tim Kelch, Dirk Holz, Adam Norton, and Stefan Schaal

TL;DR
This paper evaluates off-the-shelf industrial solutions on a new benchmark for robotic assembly, revealing their limitations and establishing a baseline for future research in robotic manipulation tasks.
Contribution
It provides the first objective benchmark performance of off-the-shelf solutions on NIST Assembly Task Boards, highlighting key challenges and limitations.
Findings
OTS solutions are expertise-dependent and have limited applicability.
Interoperability and scene awareness are lacking in current solutions.
The study establishes a baseline for future benchmarking efforts.
Abstract
In recent years, many learning based approaches have been studied to realize robotic manipulation and assembly tasks, often including vision and force/tactile feedback. However, it remains frequently unclear what is the baseline state-of-the-art performance and what are the bottleneck problems. In this work, we evaluate some off-the-shelf (OTS) industrial solutions on a recently introduced benchmark, the National Institute of Standards and Technology (NIST) Assembly Task Boards. A set of assembly tasks are introduced and baseline methods are provided to understand their intrinsic difficulty. Multiple sensor-based robotic solutions are then evaluated, including hybrid force/motion control and 2D/3D pattern matching algorithms. An end-to-end integrated solution that accomplishes the tasks is also provided. The results and findings throughout the study reveal a few noticeable factors that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Manufacturing Process and Optimization · Industrial Vision Systems and Defect Detection
