EPIC: Abstraction and Polymorphism of In-Network Collectives on Ethernet

TL;DR

EPIC introduces a standardized Ethernet-based abstraction and polymorphic implementation for in-network collective acceleration, enhancing AI training efficiency and hardware adaptability.

Contribution

It presents a novel Ethernet-compatible abstraction and modular, verified polymorphic realizations for in-network collectives, facilitating incremental hardware development and broad adoption.

Findings

Validated correctness through formal verification and extensive testing.

Achieved performance improvements in various simulation and hardware environments.

Demonstrated feasibility of the EPIC approach across multiple platforms.

Abstract

In-Network Collective (INC) acceleration holds immense potential for optimizing AI training and inference; however, its cross-layer nature has historically hindered investment and adoption within the open Ethernet ecosystem. To bridge this gap, we propose EPIC (Ethernet Polymorphic In-network Collective), an INC protocol specification and reference system built on the principle of "Unified Abstraction, Polymorphic Realization." EPIC introduces an abstraction compatible with standard Ethernet that aligns functional boundaries with participant roles, while offering polymorphic realizations tailored to varying hardware capabilities. We address three fundamental challenges: first, we employ a modular design that enables an evolutionary path from simple to complex implementations, allowing vendors to iterate their hardware incrementally; second, we apply formal verification methodologies to prove the correctness of all proposed polymorphic modes; and third, we develop a unified resource management model versatile enough for diverse INC scenarios. Extensive validation -- spanning model checking, packet/flow simulations, VM emulation, Tofino Testbed, and FPGA/RTL verification -- confirms EPIC's correctness, performance gain, and feasibility.