Multivariate Analysis for Computing Maxima in High Dimensions

TL;DR

This paper introduces a new algorithm for computing the Maxima of high-dimensional point sets that adapts to input structure, significantly improving performance for dimensions four and above.

Contribution

It presents a structure-sensitive algorithm that enhances the running time for Maxima computation in high dimensions, surpassing previous solutions.

Findings

Improved algorithm for d ≥ 4 dimensions

Runs faster on structured input sets

Outperforms existing methods for high-dimensional Maxima

Abstract

We study the problem of computing the \textsc{Maxima} of a set of $n$ $d$-dimensional points. For dimensions 2 and 3, there are algorithms to solve the problem with order-oblivious instance-optimal running time. However, in higher dimensions there is still room for improvements. We present an algorithm sensitive to the structural entropy of the input set, which improves the running time, for large classes of instances, on the best solution for \textsc{Maxima} to date for $d \ge 4$.

Equations

$$\sum _{j=0}^{\log \log n}$$

$$\subseteq \sum _{j=0}^{\log \log n}\sum _{k=1}^h\min \{n_k,\mathcal{O}(n/2^{2^j/d})\}\times 2^{j(d-2)}$$

$$=\sum _{k=1}^h\sum _{j=0}^{\log \log n}\min \{n_k,\mathcal{O}(n/2^{2^j/d})\}\times 2^{j(d-2)}\text{(reordering terms)}$$

$$\in \sum _{k=1}^h\mathcal{O}\left(\sum _{j=0}^{\log (d\log \frac{n}{n_k})}{n}_k\times 2^{j(d-2)}+\sum _{j=1+\log (d\log \frac{n}{n_k})}^{\log \log n}\frac{n}{2^{\frac{1}{d}{2}^j}}\times 2^{j(d-2)}\right)$$

$$\sum _{j=1+\log (d\log \frac{n}{n_k})}^{\log \log n}\frac{n}{2^{\frac{2^j}{d}}}\times 2^{j(d-2)}\le \sum _{j=1+\log (d\log \frac{n}{n_k})}^{\infty }\frac{n}{2^{\frac{2^{j-1}}{d}}}$$

$$\sum _{j=1+\log (d\log \frac{n}{n_k})}^{\infty }$$

$$=n_k\sum _{i=0}^{\infty }\frac{1}{(n/n_k{)}^{2^i-1}}\le n_k\sum _{i=0}^{\infty }\frac{1}{(2{)}^{2^i-1}}$$

$$\subseteq \mathcal{O}(n_k)$$

$$\mathcal{O}\left(\sum _{k=1}^h{n}_k{\log }^{d-2}\frac{n}{n_k}\right)\subseteq \mathcal{O}\left(n{\log }^{d-2}h\right)$$

$$\sum _i{a}_k\phi \left(\frac{b_k}{a_k}\right)\le a\phi \left(\frac{b}{a}\right).$$

$$\sum _{k=1}^h\frac{n_k}{n}{\log }^{d-2}\left(\frac{n}{n_k}\right)\le {\log }^{d-2}h.$$

Full text

1 Introduction

The problem of computing the Maxima of a set $\mathcal{S}$ of points was first formulated in 1974 by [9]: a point from $\mathcal{S}$ is called maximal if none of the remaining vectors in $\mathcal{S}$ dominates it in every component, and the Maxima of $\mathcal{S}$ (denoted by $M(\mathcal{S})$) is the set of maximal points in $\mathcal{S}$. For any constant dimension $d$ this problem can be solved naively in time within $\mathcal{O}(n^{2})$ by comparing every possible pair of points. [9] proposed two different algorithms to solve the problem in two and three dimensions, respectively, running in time within $\mathcal{O}(n\log n)$. For higher dimensions, [9] presented a divide-and-conquer algorithm running in time within $\mathcal{O}(n\log^{d-2}n)$ for dimensions $d>3$, and showed a lower bound of within $\Omega(n\log n)$ for this problem for any dimension $d>2$.

In 1985, [8] gave the first output-size sensitive algorithm for this problem, running in time within $\mathcal{O}(n\log h)$ in the plane, and within $\mathcal{O}(n\log^{d-2}{h})$ for dimension $d>3$, where $h$ is the size of the Maxima. In 2 and 3 dimensions, this result remained unbeaten for almost 25 years until [2] described an instance-optimal algorithm for this problem (i.e., an algorithm whose cost is at most a constant factor from the cost of any other algorithm running on the same input, for every input instance).

Given this improvement in 2 and 3 dimensions, one natural question is whether the same technique can be applied to higher dimensions in order to improve upon [8]’s results [8] in dimension $d\geq 4$. We show that the upper bound can indeed be applied to higher dimensions, even though the generalization of the order-oblivious instance-optimality result is still open due to the lack of advanced lower bound techniques in high dimensions. We introduce some basic definitions in Section 2; and describe our generalization of [2]’s upper bound [2] to high dimension in Section 3, before discussing the potential for instance-optimality results in high dimension in Section 4.

2 Preliminaries

An algorithm is said to be instance-optimal if its cost is at most a constant factor from the cost of any other algorithm on the same input, for every input instance. As such algorithm does not always exist, it is often useful to consider a relaxation of this concept, order-oblivious instance-optimality: an algorithm is said to be order-oblivious instance-optimal if it is instance-optimal on the worst input order possible.

[2] improved on the analysis of [8] for the computation of the Maxima in dimensions 2 and 3 refining it to order-oblivious instance-optimal algorithms. In 2 dimensions, they showed that a simple variant of the output sensitive algorithm originally described by [8] is, surprisingly, order-oblivious instance-optimal. For the 3 dimensional case, they described a completely new algorithm and a proof of its order-oblivious instance-optimality.

To refine the analysis, [2] introduced the notion of structural entropy of a set $\mathcal{S}$ of points, a measure of difficulty of the input instances of Maxima, and described two algorithms sensitive to this measure. To define the structural entropy of a point-set, [2] first introduced the concept of respectful partition:

Definition 1 (Respectful Partition)

A partition $\Pi$ of a set $\mathcal{S}$ of points in $\mathbb{R}^{d}$ into disjoint subsets $\mathcal{S}_{1},\ldots,\mathcal{S}_{t}$ is said to be respectful if each subset $\mathcal{S}_{k}$ is either a singleton or can be enclosed by a $d$-dimensional axis-aligned box $B_{k}$ whose interior is completely below the Maxima of $\mathcal{S}$.

Intuitively, the entropy of a respectful partition is the minimum number of bits required to encode it, and the structural entropy of a point-set is the entropy of a respectful partition with minimal entropy. Formally, the structural entropy of a point-set is defined as follows:

Definition 2 (Structural Entropy)

The entropy $\mathcal{H}(\Pi)$ of a partition $\Pi$ of $\mathcal{S}$ into disjoint subsets $\mathcal{S}_{1},\ldots,\mathcal{S}_{t}$ is defined as the value $\sum_{k=1}^{t}{(|\mathcal{S}_{k}|/n)\log(n/|\mathcal{S}_{k}|)}$. The structural entropy $\mathcal{H}(\mathcal{S})$ of the input set $\mathcal{S}$ is the minimum of $\mathcal{H}(\Pi)$ over all respectful partitions $\Pi$ of $\mathcal{S}$.

[2] showed that the Maxima of a point-set in dimensions 2 and 3, can be computed in time within $\mathcal{O}(n(\mathcal{H}(\mathcal{S})+1))$, and proved a matching lower bound for any algorithm which ignores the input order.

In Section 3 we generalize to higher dimensions [2]’s upper bound on the computational complexity of Maxima in the worst case over instances of fixed size and structural entropy. This yields an algorithm sensitive to the structural entropy of the input set, hence improving on the solution described by [8] for $d\geq 4$.

3 Multivariate analysis for the computation of Maxima

We present the algorithm DPC-Maxima (where DPC stands for “Divide, Prune, and Conquer”) which computes the Maxima of a $d$-dimensional set $\mathcal{S}$ of points with running time sensitive to the structural entropy of the input set.

Similar to [2]’s algorithm [2], DPC-Maxima performs several pruning steps removing from $\mathcal{S}$ points that are detected to be dominated, until the remaining set becomes the Maxima of the original set. For this, the algorithm repeatedly partitions $\mathcal{S}$ into an increasing number of subsets, and for each partition removes from $\mathcal{S}$ the subsets that are contained in a box completely below the Maxima of $\mathcal{S}$. The following is an outline of DPC-Maxima:

Note that DPC-Maxima always terminates: due to the restriction on the size of the subsets (in step 6), at most $\log\log n$ iterations of the outer loop will be performed for any set $S$ of $n$ points. Besides, the algorithm is correct: only dominated points are removed, and in the last iteration of the outer loop every subset is a singleton, so the points remaining in $\mathcal{S}$ after that iteration are precisely those in the Maxima of $\mathcal{S}$.

In the rest of the section we analyze the running time of DPC-Maxima. Two main issues need to be addressed: how partitions are chosen in step 6, and how the filter steps 7-9 are performed efficiently. Partitions need to be chosen with care: if $B$ is a box in an optimal respectful partition of $\mathcal{S}$, the subsets considered by the algorithm that contain points within $B$ need to be small (compact) enough so they fall completely inside $B$ (and the subsets get pruned), but large enough that only a small number of subsets is needed to prune all the points within $B$. Besides, for a given partition, DPC-Maxima needs to check in small time whether the minimum axis-aligned enclosing rectangle of each subset in the partition is below the Maxima of $S$.

We address first the selection of the partition. We show in the following lemma that any point set $S$ can be partitioned into subsets so that the faces of any axis aligned box $B$ intersects a small number of them. In particular, if $B$ is a box in a respectful partition of $S$, the lemma provides a bound on the number of points in $\mathcal{S}\cap B$ remaining after the filtering in steps 7-9 of the algorithm DPC-Maxima.

Lemma 1

Let $\mathcal{S}$ be a set of $n$ points in $\mathbb{R}^{d}$, and $r$ be an integer in $[1..n]$. There is a partition $\Pi$ of $\mathcal{S}$ into $r$ subsets such that each subset has at most $n/r$ points, and for any axis aligned $d$-dimensional box $B$:

1. i.

$B$* partially intersects (i.e. intersects but does not contain) at most $\mathcal{O}(r^{1-1/d})$ subsets in $\Pi$* 2. ii.

There are at most $\min\{|\mathcal{S}\cap B|,\mathcal{O}(n/r^{1/d})\}$ points within $B$ that belong to subsets in $\Pi$ not fully contained within $B$

Besides, this partition can be found in time within $\mathcal{O}(|\mathcal{S}|\log r)$.

Proof

The partition $\Pi$ can be obtained building a $k$-$d$ tree [4], using the median to split at each level, and recursing until the number of points within each cell drops to $n/r$ or less. Built like this, the $k$-$d$ tree will have at most $\log r$ levels and $r$ leaf cells, each containing at most $n/r$ points. For each leaf cell in the $k$-$d$ tree, the subset of points contained within it is added to $\Pi$. If needed, empty subsets are added also to obtain the total $r$ subsets. This way of proceeding requires running time within $\mathcal{O}(|\mathcal{S}|)$ for each level, for a total running time within $\mathcal{O}(|\mathcal{S}|\log r)$.

To prove proposition (i), we use the fact that any axis aligned $d$-dimensional box intersects at most $\mathcal{O}(r^{1-1/d})$ leaf cells in a $k$-$d$ tree with $\mathcal{O}(r)$ leaf cells [4]. Since the subsets in $\Pi$ fall within cells in a $k$-$d$ tree with at most $r$ leafs, the result follows. Finally, (ii) derives from (i) and from the fact that every subset has at most $n/r$ points: the points within $B$ belong, in the partition, to subsets that are completely contained within $B$, or partially intersecting $B$; the latest ones can be at most $\mathcal{O}(r^{1-1/d})$ because of (i), and contain at most $n/r$ points each, for a total number of points within $\mathcal{O}(n/r^{1/d})$, which completes the proof. ∎

Now, we turn the attention to the filtering process in steps 7-9 of the algorithm DPC-Maxima. For this, we introduce a data structure that answers dominance queries over a point set, and that benefits from knowing a priori the number of queries to perform. With it, the subsets in the partition that fall within boxes completely below the Maxima can be detected in small time.

Lemma 2

Given a set $S$ of $n$ points in $\mathbb{R}^{d}$, and an integer parameter $r\in[1..n]$, there is a data structure that answers $r$ dominance queries over $S$ (i.e. given a query point $q$, detecting whether exists a point $p\in\mathcal{S}$ that dominates $q$) in total time and space within $\mathcal{O}(n\log^{d-2}{r})$.

Proof

[2] introduced a simple data structure that answers dominance queries over a point-set of size $n$ in time within $\mathcal{O}(\log^{d-2}n)$, with preprocessing time within $\mathcal{O}(n\log^{d-2}n)$. Combining this data structure with a grouping trick (e.g. such as described by [5]) yields the result in the lemma: $S$ is partitioned into $\lceil n/r\rceil$ subsets $S_{1},\ldots,S_{\lceil n/r\rceil}$ of size at most $r$, and each $S_{i}$ is processed into a data structure to answer dominance queries over $S_{i}$. The total preprocessing time is within $\mathcal{O}(\frac{n}{r}(r\log^{d-2}r))=\mathcal{O}(n\log^{d-2}r)$. To answer whether a point $q$ is dominated by any of the points in $S$, the data structures corresponding to each $S_{i}$ are queried, for $i\in[1..\lceil n/r\rceil]$: the final answer is “yes” only if $q$ is dominated in any of the data structures. Thus, each query is answered in time within $\mathcal{O}(\frac{n}{r}\log^{d-2}r)$, for a total time within $\mathcal{O}(n\log^{d-2}{r})$. ∎

Note that, to detect whether a subset of $\mathcal{S}$ falls within a box that is completely below the Maxima of $\mathcal{S}$ (as in step 9 of DPC-Maxima), it is enough to query whether the corner with maximal coordinates (i.e. the “upper, right,…” corner) of the minimum axis-aligned rectangle of the subset is dominated by any point in $\mathcal{S}$. With this observation, and lemmas 1 and 2 we provide a first bound for the running time of DPC-Maxima:

Lemma 3

Let $\mathcal{S}$ be a set of points, and $\sigma_{j}$ be the size of $\mathcal{S}$ right after iteration $j$ of the algorithm DPC-Maxima. The running time of DPC-Maxima is within $\mathcal{O}\left(\sum_{j=0}^{\log\log n}{\sigma_{j}\times 2^{j(d-2)}}\right)$.

Proof

Let $r_{j}=2^{2^{j}}$ (as defined in step 5 of DPC-Maxima). Obtaining the partition in step 6 using Lemma 1 costs time within $\mathcal{O}(|\mathcal{S}|\log r_{j})$. Besides, the data structure described in Lemma 2 allows to perform the filtering steps in lines 7-9 collectively in total time within $\mathcal{O}(\sigma_{j}\log^{d-2}r_{j})$ (using $r_{j}$ as the parameter in the lemma). Applying the definition of $r_{j}$ yields a total time within $\mathcal{O}(\sigma_{j}\times 2^{j(d-2)})$. Since at most $\log\log n$ iterations of the outer loop are performed, the result follows. ∎

Finally, we show that DPC-Maxima runs in time sensitive to the structural entropy of the input set in the following theorem:

Theorem 3.1

Let $\mathcal{S}$ be a set of $n$ points in $\mathbb{R}^{d}$, and let $\Pi$ be any optimal respectful partition of $\mathcal{S}$ into subsets ${\mathcal{S}_{1},\ldots,\mathcal{S}_{h}}$ of sizes $n_{1},\ldots,n_{h}$, respectively. The algorithm DPC-Maxima computes the Maxima of $\mathcal{S}$ in time within $\mathcal{O}(n+\sum_{k=1}^{h}{n_{k}\log^{d-2}\frac{n}{n_{k}}})$.

Proof

Let $\sigma_{j}$ be the size of $\mathcal{S}$ during iteration $j$. From Lemma 3 we have that the running time of DPC-Maxima is within $\mathcal{O}\left(\sum_{j=0}^{\log\log n}{\sigma_{j}\times 2^{j(d-2)}}\right)$. Now, consider a subset $\mathcal{S}_{k}$ in $\Pi$, and let $B_{k}$ be the minimum axis-aligned box enclosing $\mathcal{S}_{k}$. By Lemma 1.(ii) there are at most $\min\{n_{k},\mathcal{O}(n/2^{2^{j}/d})\}$ points in $\mathcal{S}_{k}$ that remain in $\mathcal{S}$ after iteration $j$ (remember that $r_{j}=2^{2^{j}}$ is used as the parameter in the lemma). By summing over all the subsets in the partition, and plugging the previous bound for the point remaining in each subset after the $j$-th iteration of the outer loop, we have that the running time of the algorithm is within:

[TABLE]

The left inner summation can be bounded by directly by $\mathcal{O}(n_{k}\log^{d-2}\frac{n}{n_{k}})$ 111Use the identity $\sum_{i=0}^{m}{2^{c\times i}}=\frac{2^{c+cm}-1}{2^{c}-1}\in\mathcal{O}(2^{cm})\subseteq\mathcal{O}(\log^{c}m)$, for any constant $c$.. To show that the right one is within $\mathcal{O}(n_{k})$, start by bounding $2^{j(d-2)}$ by $2^{\frac{2^{j-1}}{d}}$, and take the infinite sum:

[TABLE]

This settles the case when $n_{k}>n/2$: when $n$ is factored out, the remaining series converges, and the total summation is within $\mathcal{O}(n)\subseteq\mathcal{O}(n_{k})$. For $n_{k}\leq n/2$, we substitute the variable of the sum by $i=j-\log(d\log\frac{n}{n_{k}})-1$ to obtain:

[TABLE]

Finally, replacing the bounds for the two inner summations yields the bound $\mathcal{O}\left(\sum_{k=1}^{h}{n_{k}(1+\log^{d-2}\frac{n}{n_{k}})}\right)\subseteq\mathcal{O}\left(n+\sum_{k=1}^{h}{n_{k}\log^{d-2}\frac{n}{n_{k}}}\right)$. ∎

We prove next that, thanks to the concavity of the polylog function, the bound in Theorem 3.1 is never asymptotically worse than $\mathcal{O}(n\log^{d-2}h)$, and hence the running time of DPC-Maxima is never worse than the running time of [8]’s algorithm [8]:

Lemma 4

Let $\mathcal{S}$ be a set of $n$ points in $\mathbb{R}^{d}$, of Maxima of size $h$, and let $\Pi$ be any optimal respectful partition of $\mathcal{S}$ into subsets ${\mathcal{S}_{1},\ldots,\mathcal{S}_{h}}$ of sizes $n_{1},\ldots,n_{h}$, respectively. Then,

[TABLE]

Proof

For any concave function $\varphi$, and $a=\sum_{k}a_{k}$, $b=\sum_{k}b_{k}$, Gibbs’ inequality states that

[TABLE]

Since $f(x)=\log^{d-2}x$ is a concave function 222$f(x)^{\prime\prime}=d(d-1-\log x)(log^{d-2}x)/x^{2}\leq 0$ for all $x\geq 2^{d-1}$, choosing $a_{k}=\frac{n_{k}}{n}$, $b_{k}=1$ for $k=[1..h]$ yields:

[TABLE]

Multiplying both sides of the inequality by $n$ yields the result. ∎

The upper bound in Theorem 3.1 properly generalizes [2]’s result [2] for dimensions 2 and 3 ($d\in[2,3]$), but the lower bound proving is more tricky to prove: we discuss in the next section.

4 Discussion

The bound of $\mathcal{O}(n+\sum_{k=1}^{h}{n_{k}\log^{d-2}\frac{n}{n_{k}}})$ in Theorem 3.1 is within $\Theta(n\log^{d-2}h)$ for instances where all the subsets in an optimal respectful partition have equal size $n/h$ (as illustrated in Figure 1.a). However, for instances with optimal unbalanced partitions (and hence with low structural entropy), the improvement in running time of DPC-Maxima over [8]’s algorithm [8] can be significant. Consider, for example, an instance where all the points not in the Maxima are dominated by a same unique point (as the one illustrated in Figure 1.b), and let $h\in\mathcal{O}(n^{1-\varepsilon})$ be the size of the Maxima: [8]’s algorithm [8] for this instance runs in time within $\mathcal{O}(n\log^{d-2}n)$, while DPC-Maxima runs in time within $\mathcal{O}(n)$, linear in the input size.

For the problem of Maxima in dimension $d\geq 3$, the best computational complexity lower bound known so far is $\Omega(n(\mathcal{H}(\mathcal{S})+1))=\Omega(n+\sum_{k=1}^{h}{n_{k}\log\frac{n}{n_{k}}})$, obtained by extending the bound given in 2009 by [2] for 2 dimensions. For dimension $d\geq 4$ there is a gap between this lower bound and the running time of the algorithm DPC-Maxima (which increases with $d$). The generalization of the order-oblivious instance-optimality result for the $d$-dimensional case for $d\geq 4$ is still open. In general, there is a lack of advanced lower bound techniques for problems handling high dimensional data. In this sense, a theory of fine classes of problems (as partially done in the field of Parameterized Complexity [6]) could help to palliate this lack of techniques for lower bounds.

A closely related problem is Dominance Reporting, where for a set $\mathcal{S}$ of $n$ points in $d$-dimensional space, and a set of query points $Q$, one must report the points $q\in Q$ for which there is a point in $\mathcal{S}$ that dominates $q$. For the offline version of the problem, where the size of $Q$ is known to be significantly bigger than the size of $\mathcal{S}$, the fastest data structure known to date, in dimension $d\geq 3$, was presented in 2012 by [1]. [7] considered an online version, where the size of $Q$ is not known in advance, for which they describe a deferred data structure sensitive to the sizes of $\mathcal{S}$ and $Q$. One natural question is whether the results by [7] for online Dominance Reporting can be improved via a deferred data structure sensitive to the structural entropy of the input set to query.