Decoding Non-Linearity and Complexity: Deep Tabular Learning Approaches for Materials Science

TL;DR

This paper explores deep learning models, including encoder-decoder and attention mechanisms, to better capture complex, skewed relationships in materials science tabular data, comparing their performance to traditional tree-based models.

Contribution

It introduces deep learning architectures like DNF-nets for materials data, demonstrating their effectiveness in modeling non-linear and skewed relationships beyond traditional methods.

Findings

XGBoost achieves the best loss and fastest trials.

Deep encoder-decoder models like DNF-nets perform competitively.

Deep models such as CNN have slower convergence and trial durations.

Abstract

Materials data, especially those related to high-temperature properties, pose significant challenges for machine learning models due to extreme skewness, wide feature ranges, modality, and complex relationships. While traditional models like tree-based ensembles (e.g., XGBoost, LightGBM) are commonly used for tabular data, they often struggle to fully capture the subtle interactions inherent in materials science data. In this study, we leverage deep learning techniques based on encoder-decoder architectures and attention-based models to handle these complexities. Our results demonstrate that XGBoost achieves the best loss value and the fastest trial duration, but deep encoder-decoder learning like Disjunctive Normal Form architecture (DNF-nets) offer competitive performance in capturing non-linear relationships, especially for highly skewed data distributions. However, convergence rates and trial durations for deep model such as CNN is slower, indicating areas for further optimization. The models introduced in this study offer robust and hybrid solutions for enhancing predictive accuracy in complex materials datasets.