Personalized Execution Time Optimization for the Scheduled Jobs
Yang Liu, Juan Wang, Zhengxing Chen, Ian Fox, Imani Mufti, Jason, Sukumaran, Baokun He, Xiling Sun, Feng Liang

TL;DR
This paper presents a machine learning-based approach for optimizing execution times of scheduled batch jobs across multiple product domains, improving user engagement and resource efficiency at an industrial scale.
Contribution
It introduces a novel ensemble learning method combined with a best time policy for personalized job scheduling in large-scale industrial environments.
Findings
Significant improvements in product metrics like notifications and content generation.
Effective handling of user peak time conflicts through a coordination system.
Validated on production traffic serving billions of users daily.
Abstract
Scheduled batch jobs have been widely used on the asynchronous computing platforms to execute various enterprise applications, including the scheduled notifications and the candidate pre-computation for the modern recommender systems. It is important to deliver or update the information to the users at the right time to maintain the user experience and the execution impact. However, it is challenging to provide a versatile execution time optimization solution for the user-basis scheduled jobs to satisfy various product scenarios while maintaining reasonable infrastructure resource consumption. In this paper, we describe how we apply a learning-to-rank approach plus a "best time policy" in the best time selection. In addition, we propose an ensemble learner to minimize the ranking loss by efficiently leveraging multiple streams of user activity signals in our scheduling decisions of the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDistributed and Parallel Computing Systems · Cloud Computing and Resource Management · Big Data and Digital Economy
