Representation Similarity: A Better Guidance of DNN Layer Sharing for Edge Computing without Training

TL;DR

This paper introduces a novel model merging approach for DNNs in edge computing, using representation similarity as a validation metric to guide layer sharing without requiring ground truth or retraining.

Contribution

It proposes a new merging scheme based on representation similarity, which correlates strongly with accuracy and eliminates the need for ground truth or retraining.

Findings

Representation similarity correlates highly with model accuracy (|r| > 0.94).

The proposed method guides layer sharing effectively without ground truth.

Preliminary results show promising potential for edge DNN model merging.

Abstract

Edge computing has emerged as an alternative to reduce transmission and processing delay and preserve privacy of the video streams. However, the ever-increasing complexity of Deep Neural Networks (DNNs) used in video-based applications (e.g. object detection) exerts pressure on memory-constrained edge devices. Model merging is proposed to reduce the DNNs' memory footprint by keeping only one copy of merged layers' weights in memory. In existing model merging techniques, (i) only architecturally identical layers can be shared; (ii) requires computationally expensive retraining in the cloud; (iii) assumes the availability of ground truth for retraining. The re-evaluation of a merged model's performance, however, requires a validation dataset with ground truth, typically runs at the cloud. Common metrics to guide the selection of shared layers include the size or computational cost of shared layers or representation size. We propose a new model merging scheme by sharing representations (i.e., outputs of layers) at the edge, guided by representation similarity S. We show that S is extremely highly correlated with merged model's accuracy with Pearson Correlation Coefficient |r| > 0.94 than other metrics, demonstrating that representation similarity can serve as a strong validation accuracy indicator without ground truth. We present our preliminary results of the newly proposed model merging scheme with identified challenges, demonstrating a promising research future direction.