Partitioning Strategies for Parallel Computation of Flexible Skylines

TL;DR

This paper presents a parallel computation scheme for flexible skyline queries, improving efficiency by combining parallel and sequential phases, with filtering and elimination strategies tested in PySpark.

Contribution

It introduces a novel parallel and sequential hybrid approach for flexible skyline computation, including filtering and elimination techniques to enhance performance.

Findings

Parallel scheme reduces computation time significantly.

Filtering before parallel processing decreases dataset size effectively.

Eliminating the sequential phase further improves efficiency.

Abstract

While classical skyline queries identify interesting data within large datasets, flexible skylines introduce preferences through constraints on attribute weights, and further reduce the data returned. However, computing these queries can be time-consuming for large datasets. We propose and implement a parallel computation scheme consisting of a parallel phase followed by a sequential phase, and apply it to flexible skylines. We assess the additional effect of an initial filtering phase to reduce dataset size before parallel processing, and the elimination of the sequential part (the most time-consuming) altogether. All our experiments are executed in the PySpark framework for a number of different datasets of varying sizes and dimensions.