Autonomous Aggregate Sorting in Construction and Mining via Computer Vision-Aided Robotic Arm Systems

TL;DR

This paper introduces a computer vision-aided robotic system for autonomous aggregate sorting in construction and mining, achieving high success rates and demonstrating potential for improved efficiency and safety.

Contribution

It presents an integrated robotic system with advanced perception and control techniques for precise, autonomous aggregate sorting, addressing limitations of traditional methods.

Findings

97.5% success rate in sorting experiments

Effective 3D localization and size estimation

Potential for enhanced productivity and safety

Abstract

Traditional aggregate sorting methods, whether manual or mechanical, often suffer from low precision, limited flexibility, and poor adaptability to diverse material properties such as size, shape, and lithology. To address these limitations, this study presents a computer vision-aided robotic arm system designed for autonomous aggregate sorting in construction and mining applications. The system integrates a six-degree-of-freedom robotic arm, a binocular stereo camera for 3D perception, and a ROS-based control framework. Core techniques include an attention-augmented YOLOv8 model for aggregate detection, stereo matching for 3D localization, Denavit-Hartenberg kinematic modeling for arm motion control, minimum enclosing rectangle analysis for size estimation, and hand-eye calibration for precise coordinate alignment. Experimental validation with four aggregate types achieved an average grasping and sorting success rate of 97.5%, with comparable classification accuracy. Remaining challenges include the reliable handling of small aggregates and texture-based misclassification. Overall, the proposed system demonstrates significant potential to enhance productivity, reduce operational costs, and improve safety in aggregate handling, while providing a scalable framework for advancing smart automation in construction, mining, and recycling industries.