Automated Activity Recognition of Construction Equipment Using a Data Fusion Approach

TL;DR

This paper presents a data fusion method combining audio and kinematic data for automated recognition of construction equipment activities, significantly improving accuracy over single-source approaches.

Contribution

It introduces a novel multi-source data fusion approach for activity recognition in construction equipment, enhancing reliability and accuracy in complex jobsite conditions.

Findings

Up to 25% increase in recognition accuracy with data fusion

Effective combination of audio and kinematic data for activity classification

Implementation on multiple machines demonstrating robustness

Abstract

Automated monitoring of construction operations, especially operations of equipment and machines, is an essential step toward cost-estimating, and planning of construction projects. In recent years, a number of methods were suggested for recognizing activities of construction equipment. These methods are based on processing single types of data (audio, visual, or kinematic data). Considering the complexity of construction jobsites, using one source of data is not reliable enough to cover all conditions and scenarios. To address the issue, we utilized a data fusion approach: This approach is based on collecting audio and kinematic data, and includes the following steps: 1) recording audio and kinematic data generated by machines, 2) preprocessing data, 3) extracting time- and frequency-domain-features, 4) feature-fusion, and 5) categorizing activities using a machine-learning algorithm. The proposed approach was implemented on multiple machines and the experiments show that it is possible to get up to 25% more-accurate results compared to cases of using single-data-sources.