Exploring the Impact of Serverless Computing on Peer To Peer Training Machine Learning

TL;DR

This paper proposes a novel serverless computing architecture combined with peer-to-peer networks for distributed machine learning training, significantly improving gradient computation speed at the expense of higher costs.

Contribution

It introduces a new serverless-P2P architecture for distributed training and a method for efficient parallel gradient computation under resource constraints.

Findings

Up to 97.34% improvement in gradient computation time.

Serverless architecture can cost up to 5.4 times more than traditional methods.

Dynamic resource allocation enhances training speed and resource utilization.

Abstract

The increasing demand for computational power in big data and machine learning has driven the development of distributed training methodologies. Among these, peer-to-peer (P2P) networks provide advantages such as enhanced scalability and fault tolerance. However, they also encounter challenges related to resource consumption, costs, and communication overhead as the number of participating peers grows. In this paper, we introduce a novel architecture that combines serverless computing with P2P networks for distributed training and present a method for efficient parallel gradient computation under resource constraints. Our findings show a significant enhancement in gradient computation time, with up to a 97.34\% improvement compared to conventional P2P distributed training methods. As for costs, our examination confirmed that the serverless architecture could incur higher expenses, reaching up to 5.4 times more than instance-based architectures. It is essential to consider that these higher costs are associated with marked improvements in computation time, particularly under resource-constrained scenarios. Despite the cost-time trade-off, the serverless approach still holds promise due to its pay-as-you-go model. Utilizing dynamic resource allocation, it enables faster training times and optimized resource utilization, making it a promising candidate for a wide range of machine learning applications.