AtomGPT: Atomistic Generative Pre-trained Transformer for Forward and Inverse Materials Design

TL;DR

AtomGPT is a transformer-based model tailored for materials design, capable of predicting properties and generating atomic structures, thus enabling efficient discovery and optimization of new materials.

Contribution

This work introduces AtomGPT, the first transformer model specifically designed for atomistic materials prediction and structure generation, bridging LLMs and materials science.

Findings

AtomGPT predicts material properties with accuracy comparable to graph neural networks.

It can generate atomic structures for new materials, validated by DFT calculations.

The model demonstrates potential for accelerating materials discovery.

Abstract

Large language models (LLMs) such as generative pretrained transformers (GPTs) have shown potential for various commercial applications, but their applicability for materials design remains underexplored. In this article, we introduce AtomGPT, a model specifically developed for materials design based on transformer architectures, to demonstrate the capability for both atomistic property prediction and structure generation. We show that a combination of chemical and structural text descriptions can efficiently predict material properties with accuracy comparable to graph neural network models, including formation energies, electronic bandgaps from two different methods and superconducting transition temperatures. Furthermore, we demonstrate that AtomGPT can generate atomic structures for tasks such as designing new superconductors, with the predictions validated through density functional theory calculations. This work paves the way for leveraging LLMs in forward and inverse materials design, offering an efficient approach to the discovery and optimization of materials.