Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models

TL;DR

This paper presents an automated workflow that extracts and structures materials science data from scientific literature using NLP and vision transformers, enabling faster information retrieval and knowledge synthesis.

Contribution

It introduces a novel automated pipeline combining NLP, vision transformers, and large language models to extract multi-modal data from scientific documents in materials science.

Findings

Accelerates information retrieval from scientific literature.

Enables extraction of material properties from multi-modal data.

Supports knowledge synthesis with private and unpublished data.

Abstract

To retrieve and compare scientific data of simulations and experiments in materials science, data needs to be easily accessible and machine readable to qualify and quantify various materials science phenomena. The recent progress in open science leverages the accessibility to data. However, a majority of information is encoded within scientific documents limiting the capability of finding suitable literature as well as material properties. This manuscript showcases an automated workflow, which unravels the encoded information from scientific literature to a machine readable data structure of texts, figures, tables, equations and meta-data, using natural language processing and language as well as vision transformer models to generate a machine-readable database. The machine-readable database can be enriched with local data, as e.g. unpublished or private material data, leading to knowledge synthesis. The study shows that such an automated workflow accelerates information retrieval, proximate context detection and material property extraction from multi-modal input data exemplarily shown for the research field of microstructural analyses of face-centered cubic single crystals. Ultimately, a Retrieval-Augmented Generation (RAG) based Large Language Model (LLM) enables a fast and efficient question answering chat bot.