Holistic Heterogeneous Scheduling for Autonomous Applications using Fine-grained, Multi-XPU Abstraction

TL;DR

This paper introduces XAUTO, a runtime system with a fine-grained multi-XPU abstraction that optimizes scheduling for autonomous applications, significantly reducing latency compared to existing module-level systems.

Contribution

The paper presents XNODE, a novel fine-grained multi-XPU programming abstraction, and a holistic scheduling approach that improves latency in autonomous application pipelines.

Findings

XAUTO reduces perception pipeline latency by 1.61x.

XNODE enables fine-grained, multi-XPU task management.

Holistic scheduling improves end-to-end latency.

Abstract

Modern autonomous applications are increasingly utilizing multiple heterogeneous processors (XPUs) to accelerate different stages of algorithm modules. However, existing runtime systems for these applications, such as ROS, can only perform module-level task management, lacking awareness of the fine-grained usage of multiple XPUs. This paper presents XAUTO, a runtime system designed to cooperatively manage XPUs for latency-sensitive autonomous applications. The key idea is a fine-grained, multi-XPU programming abstraction -- XNODE, which aligns with the stage-level task granularity and can accommodate multiple XPU implementations. XAUTO holistically assigns XPUs to XNODEs and schedules their execution to minimize end-to-end latency. Experimental results show that XAUTO can reduce the end-to-end latency of a perception pipeline for autonomous driving by 1.61x compared to a state-of-the-art module-level scheduling system (ROS2).