Evaluating SAP RPT-1 for Enterprise Business Process Prediction: In-Context Learning vs. Traditional Machine Learning on Structured SAP Data

TL;DR

This paper evaluates SAP's RPT-1 foundation model for enterprise data prediction tasks, comparing its in-context learning capabilities to traditional machine learning methods across various SAP business scenarios.

Contribution

It provides the first independent, practitioner-focused benchmark of RPT-1 on structured SAP data, highlighting its competitive performance and practical hybrid workflow.

Findings

RPT-1 achieves 91-96% of GBDT accuracy without training examples.

RPT-1 outperforms GBDT with limited context data (~75-100 rows).

Hybrid workflow proposed for rapid screening and targeted GBDT training.

Abstract

Tabular foundation models aim to make machine learning accessible for enterprise data without task-specific training. This paper presents the first independent evaluation of SAP's Retrieval Pretrained Transformer (RPT-1) from a practitioner perspective. RPT-1 is a compact 64.6 MB model pretrained on 1.34 TB of structured data across 3.1 million tables. We benchmark it against tuned gradient-boosted decision trees (XGBoost, LightGBM, CatBoost) on three SAP business scenarios: demand forecasting across SD/MM/PP modules, predictive data integrity in BC/MM/QM, and financial risk classification in FI/CO/AR. Across five-fold cross-validation on datasets ranging from 2,500 to 3,200 rows, RPT-1 reaches 91-96% of tuned GBDT accuracy without any training examples. The classification gap is modest at 3.6-4.1 percentage points on AUC-ROC, though regression tasks show wider gaps of 8.9-11.1 percentage points on R-squared. An interesting finding is a crossover at roughly 75-100 context rows where RPT-1 actually outperforms XGBoost under limited data. Based on these results, we propose a practical hybrid workflow: use RPT-1 for rapid screening, then train GBDT selectively where prediction accuracy justifies the effort. All experiments are reproducible through publicly available Hugging Face Spaces.