Mobile Big Data Analytics Using Deep Learning and Apache Spark

TL;DR

This paper introduces a scalable deep learning framework for mobile big data analytics using Apache Spark, enabling efficient training of complex models on large datasets with real-world validation.

Contribution

It presents a distributed deep learning approach over Spark that accelerates training of deep models on mobile big data, combining partial model averaging for scalability.

Findings

Framework significantly speeds up deep model training on large datasets.

Validation with real-world data demonstrates effective activity recognition.

Distributed approach handles millions of samples efficiently.

Abstract

The proliferation of mobile devices, such as smartphones and Internet of Things (IoT) gadgets, results in the recent mobile big data (MBD) era. Collecting MBD is unprofitable unless suitable analytics and learning methods are utilized for extracting meaningful information and hidden patterns from data. This article presents an overview and brief tutorial of deep learning in MBD analytics and discusses a scalable learning framework over Apache Spark. Specifically, a distributed deep learning is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall MBD, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.