ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modeling

TL;DR

ZENN introduces a thermodynamics-inspired neural network framework that effectively learns from heterogeneous datasets, improving generalization and robustness in classification and scientific modeling tasks.

Contribution

The paper extends zentropy theory into data science, designing a neural network that captures intrinsic entropy and energy, enabling better handling of diverse, multi-source data.

Findings

Outperforms state-of-the-art models on CIFAR-10/100, BBCNews, and AGNews datasets.

Successfully reconstructs Helmholtz energy landscape of Fe3Pt from DFT data.

Demonstrates robustness in predicting high-order derivatives.

Abstract

Traditional entropy-based methods - such as cross-entropy loss in classification problems - have long been essential tools for representing the information uncertainty and physical disorder in data and for developing artificial intelligence algorithms. However, the rapid growth of data across various domains has introduced new challenges, particularly the integration of heterogeneous datasets with intrinsic disparities. To address this, we introduce a zentropy-enhanced neural network (ZENN), extending zentropy theory into the data science domain via intrinsic entropy, enabling more effective learning from heterogeneous data sources. ZENN simultaneously learns both energy and intrinsic entropy components, capturing the underlying structure of multi-source data. To support this, we redesign the neural network architecture to better reflect the intrinsic properties and variability inherent in diverse datasets. We demonstrate the effectiveness of ZENN on classification tasks and energy landscape reconstructions, showing its superior generalization capabilities and robustness-particularly in predicting high-order derivatives. ZENN demonstrates superior generalization by introducing a learnable temperature variable that models latent multi-source heterogeneity, allowing it to surpass state-of-the-art models on CIFAR-10/100, BBCNews, and AGNews. As a practical application in materials science, we employ ZENN to reconstruct the Helmholtz energy landscape of Fe$_3$Pt using data generated from density functional theory (DFT) and capture key material behaviors, including negative thermal expansion and the critical point in the temperature-pressure space. Overall, this work presents a zentropy-grounded framework for data-driven machine learning, positioning ZENN as a versatile and robust approach for scientific problems involving complex, heterogeneous datasets.