Split Computing for Complex Object Detectors: Challenges and Preliminary Results

TL;DR

This paper explores the challenges and potential of split computing for complex R-CNN object detectors trained on large datasets, revealing that naive methods are ineffective and proposing the use of small bottlenecks to improve inference time.

Contribution

It is the first study to analyze split computing for large-scale object detectors and introduces small bottlenecks to enhance performance.

Findings

Naive split computing does not reduce inference time for R-CNN detectors.

Small bottlenecks can potentially improve split computing efficiency.

Extensive analysis of layer-wise tensor sizes and model sizes was conducted.

Abstract

Following the trends of mobile and edge computing for DNN models, an intermediate option, split computing, has been attracting attentions from the research community. Previous studies empirically showed that while mobile and edge computing often would be the best options in terms of total inference time, there are some scenarios where split computing methods can achieve shorter inference time. All the proposed split computing approaches, however, focus on image classification tasks, and most are assessed with small datasets that are far from the practical scenarios. In this paper, we discuss the challenges in developing split computing methods for powerful R-CNN object detectors trained on a large dataset, COCO 2017. We extensively analyze the object detectors in terms of layer-wise tensor size and model size, and show that naive split computing methods would not reduce inference time. To the best of our knowledge, this is the first study to inject small bottlenecks to such object detectors and unveil the potential of a split computing approach. The source code and trained models' weights used in this study are available at https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors .