Optimal Aggregation Algorithms for Middleware

TL;DR

This paper develops and analyzes instance optimal algorithms for efficiently finding top-k objects in a database with multiple sorted lists, minimizing list access costs for monotone aggregation functions.

Contribution

It introduces and evaluates several instance optimal algorithms tailored for different access cost models and demonstrates their effectiveness in minimizing data access.

Findings

Algorithms achieve instance optimality under various cost models

Proposed methods outperform naive approaches in minimizing list access

Theoretical analysis confirms optimality bounds for the algorithms

Abstract

Let D be a database of N objects where each object has m fields. The objects are given in m sorted lists (where the ith list is sorted according to the ith field). Our goal is to find the top k objects according to a monotone aggregation function t, while minimizing access to the lists. The problem arises in several contexts. In particular Fagin (JCSS 1999) considered it for the purpose of aggregating information in a multimedia database system. We are interested in instance optimality, i.e. that our algorithm will be as good as any other (correct) algorithm on any instance. We provide and analyze several instance optimal algorithms for the task, with various access costs and models.