Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledge

TL;DR

This paper compares ChatGPT and Bard in fire engineering contexts, evaluating their responses to safety-related questions, and finds ChatGPT generally performs better, highlighting the potential of chatbots to enhance fire engineering practices.

Contribution

It provides an initial comparison of two chatbots in fire engineering, demonstrating their capabilities and limitations in handling domain-specific queries.

Findings

ChatGPT outperforms Bard in fire safety questions.

Chatbots can provide rapid access to critical fire engineering information.

Potential for chatbots to transform fire engineering education and practice.

Abstract

This communication presents preliminary findings from comparing two recent chatbots, OpenAI's ChatGPT and Google's Bard, in the context of fire engineering by evaluating their responses in handling fire safety related queries. A diverse range of fire engineering questions and scenarios were created and examined, including structural fire design, fire prevention strategies, evacuation, building code compliance, and fire suppression systems (some of which resemble those commonly present in the Fire Protection exam (FPE)). The results reveal some key differences in the performance of the chatbots, with ChatGPT demonstrating a relatively superior performance. Then, this communication highlights the potential for chatbot technology to revolutionize fire engineering practices by providing instant access to critical information while outlining areas for further improvement and research. Evidently, and when it matures, this technology will likely be elemental to our engineers' practice and education.