Are LLMs Ready for Real-World Materials Discovery?

TL;DR

This paper critically examines the current limitations of Large Language Models in materials science, highlighting failure cases and proposing a framework for developing specialized, knowledge-grounded models to accelerate real-world materials discovery.

Contribution

It introduces a framework for creating Materials Science LLMs (MatSci-LLMs) grounded in domain knowledge and outlines key challenges in data extraction and model application for materials discovery.

Findings

LLMs currently struggle with complex materials science knowledge.

High-quality, multi-modal datasets are essential for effective MatSci-LLMs.

A roadmap for future applications includes knowledge bases, in-silico design, and self-driving labs.

Abstract

Large Language Models (LLMs) create exciting possibilities for powerful language processing tools to accelerate research in materials science. While LLMs have great potential to accelerate materials understanding and discovery, they currently fall short in being practical materials science tools. In this position paper, we show relevant failure cases of LLMs in materials science that reveal current limitations of LLMs related to comprehending and reasoning over complex, interconnected materials science knowledge. Given those shortcomings, we outline a framework for developing Materials Science LLMs (MatSci-LLMs) that are grounded in materials science knowledge and hypothesis generation followed by hypothesis testing. The path to attaining performant MatSci-LLMs rests in large part on building high-quality, multi-modal datasets sourced from scientific literature where various information extraction challenges persist. As such, we describe key materials science information extraction challenges which need to be overcome in order to build large-scale, multi-modal datasets that capture valuable materials science knowledge. Finally, we outline a roadmap for applying future MatSci-LLMs for real-world materials discovery via: 1. Automated Knowledge Base Generation; 2. Automated In-Silico Material Design; and 3. MatSci-LLM Integrated Self-Driving Materials Laboratories.