Agentic Operator Generation for ML ASICs

TL;DR

TritorX is an AI system that automatically generates correct and comprehensive PyTorch operator kernels for emerging hardware accelerators, emphasizing coverage and correctness over performance.

Contribution

The paper introduces TritorX, a novel agentic AI pipeline that generates extensive, correct PyTorch operator kernels for new accelerator hardware, surpassing prior limited-performance kernel approaches.

Findings

Generated kernels for 481 unique operators passing all tests

Successfully integrated with real hardware and simulation environments

Enables overnight generation of complete operator backends for new accelerators

Abstract

We present TritorX, an agentic AI system designed to generate functionally correct Triton PyTorch ATen kernels at scale for emerging accelerator platforms. TritorX integrates open-source large language models with a custom linter, JIT compilation, and a PyTorch OpInfo-based test harness. This pipeline is compatible with both real Meta Training and Inference Accelerator (MTIA) silicon and in hardware simulation environments for next-generation devices. In contrast to previous kernel-generation approaches that prioritize performance for a limited set of high-usage kernels, TritorX prioritizes coverage. Our system emphasizes correctness and generality across the entire operator set, including diverse data types, shapes, and argument patterns. In our experiments, TritorX successfully generated kernels and wrappers for 481 unique ATen operators that pass all corresponding PyTorch OpInfo tests (over 20,000 in total). TritorX paves the way for overnight generation of complete PyTorch ATen backends for new accelerator platforms.