Makespan Minimization in Split Learning: From Theory to Practice

TL;DR

This paper investigates the problem of minimizing makespan in split learning for distributed machine learning, providing complexity results, approximation algorithms, and a heuristic that improves practical performance in heterogeneous IoT environments.

Contribution

It offers the first complexity analysis for client-helper assignment in split learning, introduces a 5-approximation algorithm, and develops a heuristic for heterogeneous tasks with empirical validation.

Findings

Polynomial-time 5-approximation algorithm for homogeneous tasks

Proven intractability of optimal solutions for heterogeneous tasks

Heuristic outperforms prior methods in extensive experiments

Abstract

Split learning recently emerged as a solution for distributed machine learning with heterogeneous IoT devices, where clients can offload part of their training to computationally-powerful helpers. The core challenge in split learning is to minimize the training time by jointly devising the client-helper assignment and the schedule of tasks at the helpers. We first study the model where each helper has a memory cardinality constraint on how many clients it may be assigned, which represents the case of homogeneous tasks. Through complexity theory, we rule out exact polynomial-time algorithms and approximation schemes even for highly restricted instances of this problem. We complement these negative results with a non-trivial polynomial-time 5-approximation algorithm. Building on this, we then focus on the more general heterogeneous task setting considered by Tirana et al. [INFOCOM 2024], where helpers have memory capacity constraints and clients have variable memory costs. In this case, we prove that, unless P=NP, the problem cannot admit a polynomial-time approximation algorithm for any approximation factor. However, by adapting our aforementioned 5-approximation algorithm, we develop a novel heuristic for the heterogeneous task setting and show that it outperforms heuristics from prior works through extensive experiments.