Forecasting NCAA Basketball Outcomes with Deep Learning: A Comparative Study of LSTM and Transformer Models

TL;DR

This study compares LSTM and Transformer deep learning models for predicting NCAA basketball tournament outcomes, highlighting the impact of different architectures and loss functions on predictive accuracy and calibration.

Contribution

It introduces a comprehensive framework combining feature engineering and model evaluation for sports outcome prediction using deep learning.

Findings

Transformers with BCE achieve highest AUC of 0.8473.

LSTM with Brier loss has best probabilistic calibration.

Model choice impacts prediction accuracy and reliability.

Abstract

In this research, I explore advanced deep learning methodologies to forecast the outcomes of the 2025 NCAA Division 1 Men's and Women's Basketball tournaments. Leveraging historical NCAA game data, I implement two sophisticated sequence-based models: Long Short-Term Memory (LSTM) and Transformer architectures. The predictive power of these models is augmented through comprehensive feature engineering, including team quality metrics derived from Generalized Linear Models (GLM), Elo ratings, seed differences, and aggregated box-score statistics. To evaluate the robustness and reliability of predictions, I train each model variant using both Binary Cross-Entropy (BCE) and Brier loss functions, providing insights into classification performance and probability calibration. My comparative analysis reveals that while the Transformer architecture optimized with BCE yields superior discriminative power (highest AUC of 0.8473), the LSTM model trained with Brier loss demonstrates superior probabilistic calibration (lowest Brier score of 0.1589). These findings underscore the importance of selecting appropriate model architectures and loss functions based on the specific requirements of forecasting tasks. The detailed analytical pipeline presented here serves as a reproducible framework for future predictive modeling tasks in sports analytics and beyond.