Using Large Language Models for the Interpretation of Building Regulations

TL;DR

This paper explores the use of large language models like GPT-3.5 and GPT-4 to automatically translate building regulations into a formal, machine-readable format, enhancing automated compliance checking in construction.

Contribution

It demonstrates the feasibility of using LLMs for translating legal building requirements into LegalRuleML with few-shot learning and prompt engineering techniques.

Findings

GPT-3.5 can learn the structure of LegalRuleML with few examples.

System prompts improve the accuracy of regulation translation.

Chain-of-thought reasoning enhances model performance in legal translation tasks.

Abstract

Compliance checking is an essential part of a construction project. The recent rapid uptake of building information models (BIM) in the construction industry has created more opportunities for automated compliance checking (ACC). BIM enables sharing of digital building design data that can be used for compliance checking with legal requirements, which are conventionally conveyed in natural language and not intended for machine processing. Creating a computable representation of legal requirements suitable for ACC is complex, costly, and time-consuming. Large language models (LLMs) such as the generative pre-trained transformers (GPT), GPT-3.5 and GPT-4, powering OpenAI's ChatGPT, can generate logically coherent text and source code responding to user prompts. This capability could be used to automate the conversion of building regulations into a semantic and computable representation. This paper evaluates the performance of LLMs in translating building regulations into LegalRuleML in a few-shot learning setup. By providing GPT-3.5 with only a few example translations, it can learn the basic structure of the format. Using a system prompt, we further specify the LegalRuleML representation and explore the existence of expert domain knowledge in the model. Such domain knowledge might be ingrained in GPT-3.5 through the broad pre-training but needs to be brought forth by careful contextualisation. Finally, we investigate whether strategies such as chain-of-thought reasoning and self-consistency could apply to this use case. As LLMs become more sophisticated, the increased common sense, logical coherence, and means to domain adaptation can significantly support ACC, leading to more efficient and effective checking processes.