UniMat: Unifying Materials Embeddings through Multi-modal Learning

TL;DR

This paper explores multi-modal learning techniques to unify diverse materials science data types, demonstrating improved embeddings by aligning and fusing atomic structures, XRD patterns, and compositions for better materials analysis.

Contribution

It introduces a framework for applying multi-modal alignment and fusion techniques to materials science data, enhancing the integration of heterogeneous data modalities.

Findings

Aligning structure graphs with XRD improves data representation.

Fusing XRD and composition data yields more robust embeddings.

Multi-modal approaches outperform single-modality methods in tasks.

Abstract

Materials science datasets are inherently heterogeneous and are available in different modalities such as characterization spectra, atomic structures, microscopic images, and text-based synthesis conditions. The advancements in multi-modal learning, particularly in vision and language models, have opened new avenues for integrating data in different forms. In this work, we evaluate common techniques in multi-modal learning (alignment and fusion) in unifying some of the most important modalities in materials science: atomic structure, X-ray diffraction patterns (XRD), and composition. We show that structure graph modality can be enhanced by aligning with XRD patterns. Additionally, we show that aligning and fusing more experimentally accessible data formats, such as XRD patterns and compositions, can create more robust joint embeddings than individual modalities across various tasks. This lays the groundwork for future studies aiming to exploit the full potential of multi-modal data in materials science, facilitating more informed decision-making in materials design and discovery.