Towards Development of Automated Knowledge Maps and Databases for Materials Engineering using Large Language Models

TL;DR

This paper presents an LLM-based workflow utilizing ChatGPT and Google Gemini Pro to extract, summarize, and organize research article data into knowledge graphs, enhancing efficiency in materials engineering research and database development.

Contribution

It introduces a novel LLM-driven method for automated literature review and knowledge graph creation in materials engineering, demonstrating improved data extraction and organization capabilities.

Findings

ChatGPT achieved an F1 score of 0.40 for exact match and 0.479 for relaxed match.

Google Gemini Pro outperformed ChatGPT with an F1 score of 0.50 for exact match.

The approach facilitates high-throughput database development for materials informatics.

Abstract

In this work a Large Language Model (LLM) based workflow is presented that utilizes OpenAI ChatGPT model GPT-3.5-turbo-1106 and Google Gemini Pro model to create summary of text, data and images from research articles. It is demonstrated that by using a series of processing, the key information can be arranged in tabular form and knowledge graphs to capture underlying concepts. Our method offers efficiency and comprehension, enabling researchers to extract insights more effectively. Evaluation based on a diverse Scientific Paper Collection demonstrates our approach in facilitating discovery of knowledge. This work contributes to accelerated material design by smart literature review. The method has been tested based on various qualitative and quantitative measures of gathered information. The ChatGPT model achieved an F1 score of 0.40 for an exact match (ROUGE-1, ROUGE-2) but an impressive 0.479 for a relaxed match (ROUGE-L, ROUGE-Lsum) structural data format in performance evaluation. The Google Gemini Pro outperforms ChatGPT with an F1 score of 0.50 for an exact match and 0.63 for a relaxed match. This method facilitates high-throughput development of a database relevant to materials informatics. For demonstration, an example of data extraction and knowledge graph formation based on a manuscript about a titanium alloy is discussed.