D&A: Resource Optimisation in Personalised PageRank Computations Using Multi-Core Machines

TL;DR

This paper introduces D&A, a novel framework for resource optimization in personalised PageRank computations on multi-core machines, significantly reducing core usage while meeting time constraints.

Contribution

The paper presents a new framework D&A that determines the optimal number of cores for workload completion under time constraints, with a novel preprocessing and scaling approach.

Findings

D&A reduces core requirements by up to 73.68%.

Framework effectively handles large benchmark datasets.

Experimental results outperform baseline methods.

Abstract

Resource optimisation is commonly used in workload management, ensuring efficient and timely task completion utilising available resources. It serves to minimise costs, prompting the development of numerous algorithms tailored to this end. The majority of these techniques focus on scheduling and executing workloads effectively within the provided resource constraints. In this paper, we tackle this problem using another approach. We propose a novel framework D&A to determine the number of cores required in completing a workload under time constraint. We first preprocess a small portion of queries to derive the number of required slots, allowing for the allocation of the remaining workloads into each slot. We introduce a scaling factor in handling the time fluctuation issue caused by random functions. We further establish a lower bound of the number of cores required under this scenario, serving as a baseline for comparison purposes. We examine the framework by computing personalised PageRank values involving intensive computations. Our experimental results show that D&A surpasses the baseline, achieving reductions in the required number of cores ranging from 38.89% to 73.68% across benchmark datasets comprising millions of vertices and edges.