Enginuity: Building an Open Multi-Domain Dataset of Complex Engineering Diagrams

TL;DR

Enginuity is a pioneering large-scale, multi-domain engineering diagram dataset with detailed annotations, aimed at advancing automated diagram understanding and AI-assisted engineering tasks.

Contribution

It introduces the first comprehensive open dataset with structural annotations across multiple engineering domains for improved diagram parsing.

Findings

Enables multimodal large language models to interpret engineering diagrams

Facilitates downstream tasks like information retrieval and simulation

Breaks barriers in AI understanding of technical visual data

Abstract

We propose Enginuity - the first open, large-scale, multi-domain engineering diagram dataset with comprehensive structural annotations designed for automated diagram parsing. By capturing hierarchical component relationships, connections, and semantic elements across diverse engineering domains, our proposed dataset would enable multimodal large language models to address critical downstream tasks including structured diagram parsing, cross-modal information retrieval, and AI-assisted engineering simulation. Enginuity would be transformative for AI for Scientific Discovery by enabling artificial intelligence systems to comprehend and manipulate the visual-structural knowledge embedded in engineering diagrams, breaking down a fundamental barrier that currently prevents AI from fully participating in scientific workflows where diagram interpretation, technical drawing analysis, and visual reasoning are essential for hypothesis generation, experimental design, and discovery.