Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning

TL;DR

This paper proposes a multi-agent collaborative inference framework that compresses intermediate features and optimizes inference tasks to reduce latency and energy consumption in edge computing scenarios.

Contribution

It introduces a novel multi-agent optimization approach with feature compression and task definition for efficient DNN inference across multiple UEs.

Findings

Reduces inference latency by up to 56%.

Saves up to 72% energy consumption.

Effective in various network types.

Abstract

Recently, deploying deep neural network (DNN) models via collaborative inference, which splits a pre-trained model into two parts and executes them on user equipment (UE) and edge server respectively, becomes attractive. However, the large intermediate feature of DNN impedes flexible decoupling, and existing approaches either focus on the single UE scenario or simply define tasks considering the required CPU cycles, but ignore the indivisibility of a single DNN layer. In this paper, we study the multi-agent collaborative inference scenario, where a single edge server coordinates the inference of multiple UEs. Our goal is to achieve fast and energy-efficient inference for all UEs. To achieve this goal, we first design a lightweight autoencoder-based method to compress the large intermediate feature. Then we define tasks according to the inference overhead of DNNs and formulate the problem as a Markov decision process (MDP). Finally, we propose a multi-agent hybrid proximal policy optimization (MAHPPO) algorithm to solve the optimization problem with a hybrid action space. We conduct extensive experiments with different types of networks, and the results show that our method can reduce up to 56\% of inference latency and save up to 72\% of energy consumption.