CSRAP: Enhanced Canvas Attention Scheduling for Real-Time Mission Critical Perception

TL;DR

This paper introduces an advanced canvas attention scheduling method that allows variable canvas sizes and frame rates, significantly improving real-time perception accuracy on edge devices under strict resource constraints.

Contribution

It extends existing canvas attention scheduling by enabling variable canvas sizes and frame rates, enhancing perception performance on resource-limited edge platforms.

Findings

Higher mean average precision (mAP) achieved

Improved recall rates observed

Better quality/cost trade-offs demonstrated

Abstract

Real-time perception on edge platforms faces a core challenge: executing high-resolution object detection under stringent latency constraints on limited computing resources. Canvas-based attention scheduling was proposed in earlier work as a mechanism to reduce the resource demands of perception subsystems. It consolidates areas of interest in an input data frame onto a smaller area, called a canvas frame, that can be processed at the requisite frame rate. This paper extends prior canvas-based attention scheduling literature by (i) allowing for variable-size canvas frames and (ii) employing selectable canvas frame rates that may depart from the original data frame rate. We evaluate our solution by running YOLOv11, as the perception module, on an NVIDIA Jetson Orin Nano to inspect video frames from the Waymo Open Dataset. Our results show that the additional degrees of freedom improve the attainable quality/cost trade-offs, thereby allowing for a consistently higher mean average precision (mAP) and recall with respect to the state of the art.