Scene Graph-Guided Generative AI Framework for Synthesizing and Evaluating Industrial Hazard Scenarios

TL;DR

This paper introduces a scene graph-guided generative AI framework that synthesizes realistic hazardous workplace images from OSHA reports, aiding hazard detection model training and evaluation.

Contribution

It presents a novel method combining scene graphs and diffusion models to generate and evaluate hazardous scenarios grounded in safety reports.

Findings

VQA Graph Score outperforms CLIP and BLIP metrics

Generated hazard scenes are realistic and semantically accurate

Framework improves hazard dataset diversity and quality

Abstract

Training vision models to detect workplace hazards accurately requires realistic images of unsafe conditions that could lead to accidents. However, acquiring such datasets is difficult because capturing accident-triggering scenarios as they occur is nearly impossible. To overcome this limitation, this study presents a novel scene graph-guided generative AI framework that synthesizes photorealistic images of hazardous scenarios grounded in historical Occupational Safety and Health Administration (OSHA) accident reports. OSHA narratives are analyzed using GPT-4o to extract structured hazard reasoning, which is converted into object-level scene graphs capturing spatial and contextual relationships essential for understanding risk. These graphs guide a text-to-image diffusion model to generate compositionally accurate hazard scenes. To evaluate the realism and semantic fidelity of the generated data, a visual question answering (VQA) framework is introduced. Across four state-of-the-art generative models, the proposed VQA Graph Score outperforms CLIP and BLIP metrics based on entropy-based validation, confirming its higher discriminative sensitivity.