In context learning Foundation models for Materials Property Prediction with Small datasets

TL;DR

This paper introduces a unified in-context learning foundation model for materials property prediction that combines composition and structure features, achieving high accuracy with minimal training and aligning well with physical principles.

Contribution

The work presents a novel ICL-FM framework integrating pretrained transformers, graph neural network embeddings, and new descriptors, demonstrating superior performance and interpretability in materials property prediction.

Findings

ICL-FM outperforms state-of-the-art models on benchmark datasets.

The model achieves a 9.93% improvement in phonon frequency prediction.

ICL-FM effectively models complex phenomena like phonon scattering and mass contrast.

Abstract

Foundation models (FMs) have recently shown remarkable in-context learning (ICL) capabilities across diverse scientific domains. In this work, we introduce a unified in-context learning foundation model (ICL-FM) framework for materials property prediction that integrates both composition-based and structure-aware representations. The proposed approach couples the pretrained TabPFN transformer with graph neural network (GNN)-derived embeddings and our novel MagpieEX descriptors. MagpieEX augments traditional features with cation-anion interaction data to explicitly measure bond ionicity and charge-transfer asymmetry, capturing interatomic bonding characteristics that influence vibrational and thermal transport properties. Comprehensive experiments on the MatBench benchmark suite and a standalone lattice thermal conductivity (LTC) dataset demonstrate that ICL-FM achieves competitive or superior performance to state-of-the-art (SOTA) models with significantly reduced training costs. Remarkably, the training-free ICL-FM outperformed sophisticated SOTA GNN models in five out of six representative composition-based tasks, including a significant 9.93\% improvement in phonon frequency prediction. On the LTC dataset, the FM effectively models complex phenomena such as phonon-phonon scattering and atomic mass contrast. t-SNE analysis reveals that the FM acts as a physics-aware feature refiner, transforming raw, disjoint feature clusters into continuous manifolds with gradual property transitions. This restructured latent space enhances interpolative prediction accuracy while aligning learned representations with underlying physical laws. This study establishes ICL-FM as a generalizable, data-efficient paradigm for materials informatics.