A Reinforcement Learning-Driven Task Scheduling Algorithm for Multi-Tenant Distributed Systems

TL;DR

This paper introduces a reinforcement learning-based task scheduling algorithm for multi-tenant distributed systems that dynamically optimizes latency, resource utilization, and fairness, outperforming existing methods in various scenarios.

Contribution

It presents a novel adaptive scheduling framework using PPO that models the process as a Markov decision process for real-time, multi-objective optimization in complex distributed environments.

Findings

Outperforms existing scheduling approaches in multiple metrics

Demonstrates strong stability and generalization in diverse scenarios

Effectively balances latency, resource efficiency, and fairness

Abstract

This paper addresses key challenges in task scheduling for multi-tenant distributed systems, including dynamic resource variation, heterogeneous tenant demands, and fairness assurance. An adaptive scheduling method based on reinforcement learning is proposed. By modeling the scheduling process as a Markov decision process, the study defines the state space, action space, and reward function. A scheduling policy learning framework is designed using Proximal Policy Optimization (PPO) as the core algorithm. This enables dynamic perception of complex system states and real-time decision-making. Under a multi-objective reward mechanism, the scheduler jointly optimizes task latency, resource utilization, and tenant fairness. The coordination between the policy network and the value network continuously refines the scheduling strategy. This enhances overall system performance. To validate the effectiveness of the proposed method, a series of experiments were conducted in multi-scenario environments built using a real-world public dataset. The experiments evaluated task latency control, resource efficiency, policy stability, and fairness. The results show that the proposed method outperforms existing scheduling approaches across multiple evaluation metrics. It demonstrates strong stability and generalization ability. The proposed scheduling framework provides practical and engineering value in policy design, dynamic resource modeling, and multi-tenant service assurance. It effectively improves scheduling efficiency and resource management in distributed systems under complex conditions.