Edge Collaborative Gaussian Splatting with Integrated Rendering and Communication

TL;DR

This paper introduces ECO-GS, a collaborative Gaussian splatting framework that optimizes rendering quality and resource use on edge devices through integrated decision-making and communication strategies.

Contribution

It proposes a novel joint optimization method for collaborative Gaussian splatting that balances fidelity and timeliness on edge devices, with efficient algorithms for real-time deployment.

Findings

The PMM algorithm effectively solves the nonconvex optimization problem.

ILO reduces computational time by over 100 times compared to PMM.

Experiments show real-time performance and improved rendering quality.

Abstract

Gaussian splatting (GS) struggles with degraded rendering quality on low-cost devices. To address this issue, we present edge collaborative GS (ECO-GS), where each user can switch between a local small GS model to guarantee timeliness and a remote large GS model to guarantee fidelity. However, deciding how to engage the large GS model is nontrivial, due to the interdependency between rendering requirements and resource conditions. To this end, we propose integrated rendering and communication (IRAC), which jointly optimizes collaboration status (i.e., deciding whether to engage large GS) and edge power allocation (i.e., enabling remote rendering) under communication constraints across different users by minimizing a newly-derived GS switching function. Despite the nonconvexity of the problem, we propose an efficient penalty majorization minimization (PMM) algorithm to obtain the critical point solution. Furthermore, we develop an imitation learning optimization (ILO) algorithm, which reduces the computational time by over 100x compared to PMM. Experiments demonstrate the superiority of PMM and the real-time execution capability of ILO.