DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation

TL;DR

This paper introduces Corki, a co-designed algorithm-architecture framework that enhances real-time robotic manipulation by reducing inference frequency and accelerating control, leading to faster and more successful robot actions.

Contribution

Corki's novel co-design decouples inference, control, and communication, predicting trajectories instead of single frames, significantly improving speed and success rates in embodied AI manipulation.

Findings

Inference frequency reduced by up to 5.1 times

Achieved up to 5.9 times speedup in robotic control

Success rate improved by up to 13.9%

Abstract

Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks. In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes \textsc{Corki}\xspace, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications. We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline. Instead of predicting action for one single frame, \textsc{Corki}\xspace predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. \textsc{Corki}\xspace largely reduces LLM inference frequency by up to $5.1\times$, resulting in up to $5.9\times$ speed up. The success rate improvement can be up to 13.9\%.