Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs

TL;DR

This paper evaluates NVIDIA's CUDA Tile abstraction across different GPU architectures and workloads, highlighting its performance benefits and portability limitations compared to other approaches.

Contribution

It provides the first independent, cross-architecture performance evaluation of CuTile on Hopper and Blackwell GPUs for AI workloads.

Findings

CuTile achieves up to 1007 TFLOP/s on Blackwell for fused attention.

CuTile reaches 52-79% of cuBLAS performance for GEMM with minimal code.

Triton demonstrates more consistent portability, maintaining 62-101% of cuBLAS performance.

Abstract

NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs. We present the first independent, cross-architecture evaluation of CuTile against established approaches such as cuBLAS, Triton, WMMA, and raw SIMT on three NVIDIA GPUs spanning Hopper and Blackwell: H100 NVL, B200, and RTX PRO 6000 Blackwell Server Edition. We benchmark representative AI workloads, including GEMM, fused multi-head attention, and end-to-end LLM inference in BF16/FP16 precision, to assess both performance and portability. Our results show that CuTile effectiveness is strongly workload- and architecture-dependent. On datacenter-class Blackwell (B200), CuTile achieves up to 1007 TFLOP/s for fused attention, outperforming FlashAttention-2 by 2.5x while requiring only 60 lines of Python kernel code. For GEMM, CuTile reaches 52-79% of cuBLAS performance in 22 lines of code (versus 123 for WMMA), making it a practical replacement for hand-written CUDA kernels but not yet for vendor-optimized libraries. However, the same CuTile attention kernel achieves only 53% of FlashAttention-2 throughput on RTX PRO 6000 (sm_120), exposing significant cross-architecture optimization gaps. In contrast, Triton sustains 62-101% of cuBLAS performance across all tested platforms without architecture-specific tuning, demonstrating substantially stronger portability.