Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction

TL;DR

This paper introduces a novel transformer-based architecture with triple attention mechanisms for accurately predicting time-dependent concrete creep, outperforming traditional models and enabling real-time, interpretable predictions in engineering.

Contribution

The paper develops a triple attention transformer model that effectively captures sequential and material interactions in concrete creep prediction, advancing beyond existing empirical and machine learning methods.

Findings

Achieves mean absolute percentage error of 1.63% and R2 of 0.999 on experimental datasets.

Demonstrates the importance of attention mechanisms, especially attention pooling, for model performance.

Identifies Young's modulus as the most influential feature for creep prediction.

Abstract

This paper presents a novel Triple Attention Transformer Architecture for predicting time-dependent concrete creep, addressing fundamental limitations in current approaches that treat time as merely an input parameter rather than modeling the sequential nature of deformation development. By transforming concrete creep prediction into an autoregressive sequence modeling task similar to language processing, our architecture leverages the transformer's self-attention mechanisms to capture long-range dependencies in historical creep patterns. The model implements a triple-stream attention framework incorporating temporal attention for sequential progression, feature attention for material property interactions, and batch attention for inter-sample relationships. Evaluated on experimental datasets with standardized daily measurements spanning 160 days, the architecture achieves exceptional performance with mean absolute percentage error of 1.63% and R2 values of 0.999 across all datasets, substantially outperforming traditional empirical models and existing machine learning approaches. Ablation studies confirm the critical role of attention mechanisms, with attention pooling contributing most significantly to model performance. SHAP analysis reveals Young's modulus as the primary predictive feature, followed by density and compressive strength, providing interpretability essential for engineering applications. A deployed web-based interface facilitates practical implementation, enabling real-time predictions using standard laboratory parameters. This work establishes the viability of applying transformer architectures to materials science problems, demonstrating the potential for data-driven approaches to revolutionize structural behavior prediction and engineering design practices.