Efficient Multi-user Offloading of Personalized Diffusion Models: A DRL-Convex Hybrid Solution

TL;DR

This paper introduces a hybrid DRL-convex optimization framework for efficient multi-user offloading of personalized diffusion models, balancing latency and accuracy on edge devices with diverse resources.

Contribution

It proposes a novel multi-user hybrid inference method and formulates the offloading problem as a GQAP extension, solved via a DRL-convex hybrid approach.

Findings

Outperforms traditional methods in optimality and complexity

Effectively balances latency and accuracy for multiple users

Reduces storage and computational burden on edge servers

Abstract

With the impressive generative capabilities of diffusion models, personalized content synthesis has emerged as the most highly anticipated. However, the large model sizes and iterative nature of inference make it difficult to deploy personalized diffusion models broadly on local devices with varying computational power. To this end, we propose a novel framework for efficient multi-user offloading of personalized diffusion models, given a variable number of users, diverse user computational capabilities, and fluctuating available computational resources on the edge server. To enhance computational efficiency and reduce storage burden on edge servers, we first propose a tailored multi-user hybrid inference manner, where the inference process for each user is split into two phases with an optimizable split point. The initial phase of inference is processed on a cluster-wide model using batching techniques, generating low-level semantic information corresponding to each user's prompt. Then, the users employ their own personalized model to add further details in the later inference phase. Given the constraints on edge server computational resources and users' preferences for low latency and high accuracy, we model the joint optimization of each user's offloading request handling and split point as an extension of the Generalized Quadratic Assignment Problem (GQAP). Our objective is to maximize a comprehensive metric that accounts for both latency and accuracy across all users. To tackle this NP-hard problem, we transform the GQAP into an adaptive decision sequence, model it as a Markov decision process, and develop a hybrid solution combining deep reinforcement learning with convex optimization techniques. Simulation results validate the effectiveness of our framework, demonstrating superior optimality and low complexity compared to traditional methods.