Quo Vadis ChatGPT? From Large Language Models to Large Knowledge Models

TL;DR

This paper discusses the limitations of large language models like ChatGPT in scientific domains and proposes the development of hybrid AI systems called Large Knowledge Models (LKMs) that integrate domain-specific knowledge for better reasoning and explanation in chemical engineering.

Contribution

It introduces the concept of Large Knowledge Models (LKMs) as an extension of LLMs, emphasizing their potential in scientific and engineering applications by integrating fundamental domain knowledge.

Findings

LLMs excel in tasks like text summarization and code assistance.

They lack reasoning and domain understanding in scientific fields.

Hybrid models combining first principles and data are promising for engineering.

Abstract

The startling success of ChatGPT and other large language models (LLMs) using transformer-based generative neural network architecture in applications such as natural language processing and image synthesis has many researchers excited about potential opportunities in process systems engineering (PSE). The almost human-like performance of LLMs in these areas is indeed very impressive, surprising, and a major breakthrough. Their capabilities are very useful in certain tasks, such as writing first drafts of documents, code writing assistance, text summarization, etc. However, their success is limited in highly scientific domains as they cannot yet reason, plan, or explain due to their lack of in-depth domain knowledge. This is a problem in domains such as chemical engineering as they are governed by fundamental laws of physics and chemistry (and biology), constitutive relations, and highly technical knowledge about materials, processes, and systems. Although purely data-driven machine learning has its immediate uses, the long-term success of AI in scientific and engineering domains would depend on developing hybrid AI systems that use first principles and technical knowledge effectively. We call these hybrid AI systems Large Knowledge Models (LKMs), as they will not be limited to only NLP-based techniques or NLP-like applications. In this paper, we discuss the challenges and opportunities in developing such systems in chemical engineering.