vAccSOL: Efficient and Transparent AI Vision Offloading for Mobile Robots

TL;DR

vAccSOL is a framework that optimizes and transparently offloads AI vision workloads for mobile robots, significantly reducing power consumption and increasing frame rates by leveraging hardware-optimized inference and edge offloading.

Contribution

It introduces vAccSOL, combining a neural network compiler and a lightweight execution framework for efficient, flexible AI vision workload offloading on heterogeneous robotic platforms.

Findings

Reduces robot power consumption by up to 80%.

Increases vision pipeline frame rate by up to 24x.

Achieves comparable or better inference performance than baseline.

Abstract

Mobile robots are increasingly deployed for inspection, patrol, and search-and-rescue operations, relying on computer vision for perception, navigation, and autonomous decision-making. However, executing modern vision workloads onboard is challenging due to limited compute resources and strict energy constraints. While some platforms include embedded accelerators, these are typically tied to proprietary software stacks, leaving user-defined workloads to run on resource-constrained companion computers. We present vAccSOL, a framework for efficient and transparent execution of AI-based vision workloads across heterogeneous robotic and edge platforms. vAccSOL integrates two components: SOL, a neural network compiler that generates optimized inference libraries with minimal runtime dependencies, and vAccel, a lightweight execution framework that transparently dispatches inference locally on the robot or to nearby edge infrastructure. This combination enables hardware-optimized inference and flexible execution placement without requiring modifications to robot applications. We evaluate vAccSOL on a real-world testbed with a commercial quadruped robot and twelve deep learning models covering image classification, video classification, and semantic segmentation. Compared to a PyTorch compiler baseline, SOL achieves comparable or better inference performance. With edge offloading, vAccSOL reduces robot-side power consumption by up to 80% and edge-side power by up to 60% compared to PyTorch, while increasing vision pipeline frame rate by up to 24x, extending the operating lifetime of battery-powered robots.