DeRelayL: Sustainable Decentralized Relay Learning

TL;DR

DeRelayL introduces a decentralized relay learning system enabling common users to collaboratively train and share machine learning models sustainably without relying on centralized entities.

Contribution

The paper proposes a novel decentralized relay learning paradigm, including architecture, incentive mechanisms, and theoretical validation, to empower widespread collaborative model training.

Findings

DeRelayL effectively facilitates sustainable collaborative model training.

The incentive mechanisms promote participant engagement and system sustainability.

Theoretical analysis and simulations confirm the system's effectiveness.

Abstract

In the era of big data, large-scale machine learning models have revolutionized various fields, driving significant advancements. However, large-scale model training demands high financial and computational resources, which are only affordable by a few technological giants and well-funded institutions. In this case, common users like mobile users, the real creators of valuable data, are often excluded from fully benefiting due to the barriers, while the current methods for accessing large-scale models either limit user ownership or lack sustainability. This growing gap highlights the urgent need for a collaborative model training approach, allowing common users to train and share models. However, existing collaborative model training paradigms, especially federated learning (FL), primarily focus on data privacy and group-based model aggregation. To this end, this paper intends to address this issue by proposing a novel training paradigm named decentralized relay learning (DeRelayL), a sustainable learning system where permissionless participants can contribute to model training in a relay-like manner and share the model. In detail, this paper presents the architecture and workflow of DeRelayL, designs incentive mechanisms to ensure sustainability, and conducts theoretical analysis and numerical simulations to demonstrate its effectiveness.