ErgoChat: a Visual Query System for the Ergonomic Risk Assessment of Construction Workers

TL;DR

This paper presents ErgoChat, an interactive visual query system utilizing vision-language models to assess ergonomic risks for construction workers, providing accurate, descriptive, and interactive feedback to improve traditional risk assessment methods.

Contribution

The study introduces a novel interactive visual query system for ergonomic risk assessment that leverages vision-language models and a specialized dataset, surpassing existing generic approaches.

Findings

VQA accuracy of 96.5% achieved

System outperforms generic dataset-trained methods

Effective in generating textual descriptions of ergonomic risks

Abstract

In the construction sector, workers often endure prolonged periods of high-intensity physical work and prolonged use of tools, resulting in injuries and illnesses primarily linked to postural ergonomic risks, a longstanding predominant health concern. To mitigate these risks, researchers have applied various technological methods to identify the ergonomic risks that construction workers face. However, traditional ergonomic risk assessment (ERA) techniques do not offer interactive feedback. The rapidly developing vision-language models (VLMs), capable of generating textual descriptions or answering questions about ergonomic risks based on image inputs, have not yet received widespread attention. This research introduces an interactive visual query system tailored to assess the postural ergonomic risks of construction workers. The system's capabilities include visual question answering (VQA), which responds to visual queries regarding workers' exposure to postural ergonomic risks, and image captioning (IC), which generates textual descriptions of these risks from images. Additionally, this study proposes a dataset designed for training and testing such methodologies. Systematic testing indicates that the VQA functionality delivers an accuracy of 96.5%. Moreover, evaluations using nine metrics for IC and assessments from human experts indicate that the proposed approach surpasses the performance of a method using the same architecture trained solely on generic datasets. This study sets a new direction for future developments in interactive ERA using generative artificial intelligence (AI) technologies.