Scalable Smartphone Cluster for Deep Learning

TL;DR

This paper introduces a scalable wired smartphone cluster with 138 devices for deep learning training, achieving significant speed-ups and overcoming wireless communication limitations of small clusters.

Contribution

It proposes a large-scale wired smartphone cluster for deep learning, surpassing wireless limitations and demonstrating high training efficiency.

Findings

Achieved 90% of P100 speed on ResNet-50

Realized 43x speed-up of V100 on MobileNet-v1

Connected 138 devices via Ethernet for scalable training

Abstract

Various deep learning applications on smartphones have been rapidly rising, but training deep neural networks (DNNs) has too large computational burden to be executed on a single smartphone. A portable cluster, which connects smartphones with a wireless network and supports parallel computation using them, can be a potential approach to resolve the issue. However, by our findings, the limitations of wireless communication restrict the cluster size to up to 30 smartphones. Such small-scale clusters have insufficient computational power to train DNNs from scratch. In this paper, we propose a scalable smartphone cluster enabling deep learning training by removing the portability to increase its computational efficiency. The cluster connects 138 Galaxy S10+ devices with a wired network using Ethernet. We implemented large-batch synchronous training of DNNs based on Caffe, a deep learning library. The smartphone cluster yielded 90% of the speed of a P100 when training ResNet-50, and approximately 43x speed-up of a V100 when training MobileNet-v1.