MTFM: A Scalable and Alignment-free Foundation Model for Industrial Recommendation in Meituan

TL;DR

This paper introduces MTFM, a scalable, alignment-free transformer-based foundation model for industrial recommendation systems that efficiently integrates multi-scenario data without strict input alignment, achieving significant performance improvements.

Contribution

MTFM is the first to transform cross-domain data into heterogeneous tokens for alignment-free multi-scenario recommendation, with novel system-level optimizations for scalability.

Findings

Significant performance gains in offline and online tests.

Enhanced training throughput via user-level sample aggregation.

Reduced memory and computational costs through specialized attention mechanisms.

Abstract

Industrial recommendation systems typically involve multiple scenarios, yet existing cross-domain (CDR) and multi-scenario (MSR) methods often require prohibitive resources and strict input alignment, limiting their extensibility. We propose MTFM (Meituan Foundation Model for Recommendation), a transformer-based framework that addresses these challenges. Instead of pre-aligning inputs, MTFM transforms cross-domain data into heterogeneous tokens, capturing multi-scenario knowledge in an alignment-free manner. To enhance efficiency, we first introduce a multi-scenario user-level sample aggregation that significantly enhances training throughput by reducing the total number of instances. We further integrate Grouped-Query Attention and a customized Hybrid Target Attention to minimize memory usage and computational complexity. Furthermore, we implement various system-level optimizations, such as kernel fusion and the elimination of CPU-GPU blocking, to further enhance both training and inference throughput. Offline and online experiments validate the effectiveness of MTFM, demonstrating that significant performance gains are achieved by scaling both model capacity and multi-scenario training data.