A Hybrid Solution to improve Iteration Efficiency in the Distributed Learning

TL;DR

This paper proposes a hybrid method for distributed machine learning that enhances iteration efficiency and fault tolerance by abandoning slow nodes' results, balancing performance and accuracy.

Contribution

It introduces a novel hybrid approach that improves iteration efficiency and fault tolerance in distributed learning systems by selectively abandoning slow nodes.

Findings

Reduces calculation time significantly.

Maintains acceptable accuracy levels.

Effective across various platforms.

Abstract

Currently, many machine learning algorithms contain lots of iterations. When it comes to existing large-scale distributed systems, some slave nodes may break down or have lower efficiency. Therefore traditional machine learning algorithm may fail because of the instability of distributed system.We presents a hybrid approach which not only own a high fault-tolerant but also achieve a balance of performance and efficiency.For each iteration, the result of slow machines will be abandoned. Then, we discuss the relationship between accuracy and abandon rate. Next we debate the convergence speed of this process. Finally, our experiments demonstrate our idea can dramatically reduce calculation time and be used in many platforms.