Smallest k-Enclosing Rectangle Revisited

TL;DR

This paper introduces faster algorithms for finding the smallest axis-aligned rectangle enclosing k points, including near quadratic, near linear, and approximation algorithms, along with complexity bounds and related problem variants.

Contribution

It presents the first near quadratic time algorithm, a k-sensitive algorithm, and a near linear approximation for the k-enclosing rectangle problem, advancing computational geometry methods.

Findings

First near quadratic time algorithm for the problem.

A k-sensitive algorithm with near O(nk) time complexity.

A near linear time (1+ε)-approximation algorithm for minimum area.

Abstract

Given a set of $n$ points in the plane, and a parameter $k$, we consider the problem of computing the minimum (perimeter or area) axis-aligned rectangle enclosing $k$ points. We present the first near quadratic time algorithm for this problem, improving over the previous near-$O(n^{5/2})$-time algorithm by Kaplan etal [KRS17]. We provide an almost matching conditional lower bound, under the assumption that $(\min,+)$-convolution cannot be solved in truly subquadratic time. Furthermore, we present a new reduction (for either perimeter or area) that can make the time bound sensitive to $k$, giving near $O(n k) $ time. We also present a near linear time $(1+\varepsilon)$-approximation algorithm to the minimum area of the optimal rectangle containing $k$ points. In addition, we study related problems including the $3$-sided, arbitrarily oriented, weighted, and subset sum versions of the problem.

Figures

Equations

$$T(n,q)=4T(n,q/2)+O(q^2),$$

$$[\frac{m}{2^i}+\frac{j}{2^{i+E}}+s,\frac{m}{2^i}+\frac{j^{\prime }}{2^{i+E}}+s]\times [-j^{\prime \prime }\epsilon p_y,p_y]$$

$$w^{*}+O({\varepsilon}){\alpha}/p_{y}\leq\bigl{(}1+O({\varepsilon})\bigr{)}{\alpha}/h^{*},$$

$$\mathrm{opt}\left(P,k\right)=\underset{\tau }{\min }\mathrm{opt}\left(P_{\tau },k_{\tau }\right).$$

$$T(n)\>\leq\>(4\ln b+1)\,T(20n/b)+O\bigl{(}(1/{\varepsilon})^{3}\log(1/{\varepsilon})\cdot n\log n\bigr{)},$$

$$O((n/q)\cdot ((n/q)T_{\text{mbox}\mathrm{convol}}(q)+q^3)).$$

$$O((n/q{)}^2\cdot (nq\log n+q^3\log n)).$$

$$O\bigl{(}(n/q)^{2}\cdot(kq\log k+q^{3}\log k)\bigr{)}.$$

$$\forall \ell :c_{\ell }\le \underset{i+j=\ell }{\min }(a_i+b_j).$$

$$\{(-i-a_i,0){\}}_{i=0}^{n-1}\cup \{(j+b_j,0){\}}_{j=0}^{n-1}\cup \{(0,n-\ell -c_{\ell }){\}}_{\ell =0}^{n-1},$$

$$\underset{i,j,\ell :(i+1)+(j+1)+(n-\ell )+2M=k}{\min }2(i+a_i+j+b_j+n-\ell -c_{\ell })=\underset{i,j,\ell :i+j=\ell }{\min }2(n+a_i+b_j-c_{\ell }),$$

$$\underset{i,j,\ell :(i+1)+(j+1)+(n-\ell )+2M=k}{\min }\frac{i+a_i+j+b_j}{\ell +c_{\ell }}=\underset{i,j,\ell :i+j=\ell }{\min }\frac{\ell +a_i+b_j}{\ell +c_{\ell }},$$

$$\{(-i,0;a_i-a_{i-1}){\}}_{i=0}^{n-1}\cup \{(j,0;b_j-b_{j-1}){\}}_{j=0}^{n-1}\cup \{(0,n-\ell ;-c_{\ell }+c_{\ell +1}){\}}_{\ell =0}^{n-1},$$

$$\underset{i,j,\ell :(i+1)+(j+1)+(n-\ell )+2M=k}{\min }(a_i+b_j-c_{\ell })=\underset{i,j,\ell :i+j=\ell }{\min }(a_i+b_j-c_{\ell }),$$

Full text

1 Introduction

Given a set $P$ of $n$ points in the plane, and a parameter $k$, consider the problem of computing the smallest area/perimeter axis-aligned rectangle that contains $k$ points of $P$. (Unless stated otherwise, rectangles are axis-aligned by default.) This problem and its variants have a long history. Eppstein and Erickson [EE94] studied an exhaustive number of variants of this problem for various shapes.

For the minimum perimeter variant, the first work on this problem seems to be Aggarwal et al. [AIKS91], who showed a brute force algorithm with running time $O(n^{3})$. Recently, Kaplan et al. [KRS17] gave an algorithm with running time $O(n^{5/2}\log^{2}n)$ that works for both minimum perimeter and area.

Several works derived algorithms with running time sensitive to $k$, the number of points in the shape. Aggarwal et al. [AIKS91] showed an algorithm for the minimum perimeter with running time $O(k^{2}n\log n)$. This was improved to $O(n\log n+k^{2}n)$ by Eppstein and Erickson [EE94] or alternatively by Datta et al. [DLSS95]. Kaplan et al.’s algorithm [KRS17] for the $k$-insensitive case, coupled with these previous techniques [EE94, DLSS95], results in an $O(n\log n+nk^{3/2}\log^{2}k)$ running time, which is currently the state of the art.

Known techniques [EE94, DLSS95] reduce the problem to solving $O(n/k)$ instances of size $O(k)$. These reductions work only for the perimeter case, not the area case – in particular, there are incorrect attributions in the literature to results on the minimum area rectangle – see the introduction of de Berg et al. [dBCC*+*16] for details. De Berg et al. described an algorithm with running time $O(n\log^{2}n+nk^{2}\log n)$ for minimum area. Both de Berg et al. [dBCC*+*16] and Kaplan et al. [KRS17] left as an open question whether there is a reduction from the minimum-area problem to about $\widetilde{O}(n/k)$ instances of size $O(k)$, where $\widetilde{O}$ hides111We reserve the right, in the future, to use the $\widetilde{O}$ to hide any other things we do not like. polynomial factors in $\log n$ and $1/{\varepsilon}$. Such a reduction would readily imply an improved algorithm.

Our results.

We revisit the above problems and provide significantly improved algorithms:

1. (A)

Exact smallest $k$-enclosing rectangle. In Section 2.1 we describe an algorithm for the minimum $k$-enclosing rectangle (either area or perimeter) with running time $O(n^{2}\log n)$ (see Theorem 2.2). It is based on a new divide-and-conquer approach, which is arguably simpler than Kaplan et al.’s algorithm. Known reductions mentioned above then lead to an $O(n\log n+nk\log k)$-time algorithm for computing the minimum perimeter rectangle. 2. (B)

$k$-sensitive running time for smallest area. In Section 2.2 we describe a reduction of the minimum-area problem to $O(\tfrac{n}{k}\log\tfrac{n}{k})$ instances of size $O(k)$ (see Theorem 2.8). Our reduction uses shallow cutting for 3-sided rectangular ranges [JL11] (see Appendix A) and is conceptually simple.

Plugging this the aforementioned new $O(n^{2}\log n)$-time algorithm leads to $O(nk\log\tfrac{n}{k}\log k)$-time algorithm for computing the minimum area $k$-enclosing rectangle (see Corollary 2.9). Thus, our new result strictly improves upon both Kaplan et al.’s and de Berg et al.’s results for all $k$, from constant to $\Theta(n)$.

The smallest enclosing rectangle problem is amenable to sampling. Kaplan et al. used samples in an approximation algorithm, with running time $\widetilde{O}(n/k)$, that computes a rectangle containing at least $(1-{\varepsilon})k$ points of a prescribed perimeter, where $k$ is the maximum number of points in any such rectangle. Similarly, using relative approximations [HS11], de Berg et al. [dBCC*+*16] showed an algorithm that computes, in $\widetilde{O}(n)$ time, a rectangle containing $\geq(1-{\varepsilon})k$ points, where $k$ is the maximum number of points in any rectangle of a prescribed area. The “dual” problem, of approximating the minimum area rectangle containing $k$ points seems harder, since sampling does not directly apply to it.

1. (C)

Approximating the area of the smallest $k$-enclosing rectangle. In Section 2.3, we present an approximation algorithm that computes, in $O(n\log n)$ expected time, a rectangle containing $k$ points of area $\leq(1+{\varepsilon})\alpha^{*}$, for a constant ${\varepsilon}\in(0,1)$, where $\alpha^{*}$ is the smallest-area of such a rectangle (see Theorem 2.13).

We next present a flotilla of related results:

1. (D)

$3$-sided smallest $k$-enclosing rectangle. In Section 3.1 we (slightly) speed up the exact algorithm for the $3$-sided rectangles case (i.e., rectangles that must have their bottom edge on the $x$-axis). The running time is O\bigl{(}n^{2}/2^{\Omega(\sqrt{\log n})}\bigr{)}, and is obtained using known results on the (min,+)-convolution problem [BCD*+*14, Wil14] (see Theorem 3.3). 2. (E)

Arbitrarily oriented smallest $k$-enclosing rectangle. In Section 3.2 we briefly consider the variant where the rectangle may not be axis-aligned. We show that this problem can be solved in $O(n^{3}\log n+n^{3}k/2^{\Omega(\sqrt{\log k})})$ time, slightly improving a previous result of $O(n^{3}k)$ [DGN05] when $k$ is not too small. 3. (F)

Minimum-weight $k$-enclosing rectangle. In Section 3.3 we show how to extend our $O(n^{2}\log n)$-time algorithm to the related problem of finding a minimum-weight rectangle that contains $k$ points, for $n$ given weighted points in the plane (see Theorem 3.4). 4. (G)

Subset sum for $k$-enclosing rectangle. In Section 3.4, we study the problem of finding a rectangle that contains $k$ points and has a prescribed weight $W$ (or as close as one can get to it). The running time of the new algorithm is $O(n^{5/2}\log n)$ (see Theorem 3.6). 5. (H)

Conditional lower bound. In Section 3.5, we prove that our near quadratic algorithm for exact minimum (perimeter or area) $k$-enclosing rectangle is near optimal up to an arbitrarily small polynomial factor, under a “popular” conjecture that the (min,+)-convolution problem cannot be solved in truly subquadratic time [CMWW17].

2 Smallest $k$-enclosing rectangle

2.1 An exact near-quadratic algorithm

Our $O(n^{2}\log n)$-time algorithm for minimum $k$-enclosing rectangles is based on divide-and-conquer. It has some similarity with an $O(n^{2})$-time divide-and-conquer algorithm by Barbay et al. [BCNP14] for a different problem (finding the minimum-weight rectangle for $n$ weighted points in the plane, without any $k$-enclosing constraint), but the new algorithm requires more ingenuity.

We start with a semi-dynamic data structure for a 1D subproblem:

Lemma 2.1**.**

Given a set $P$ of $n$ points in 1D with $q$ marked points, and an integer $k$, we can maintain an $O(q^{2})$-space data structure, with $O(n\log n+nq)$ preprocessing time, that supports the following operations:

• •

report the shortest interval containing $k$ points of $P$ in $O(q)$ time;

• •

delete a marked point in $O(q)$ time;

• •

unmark a marked point in $O(q)$ time.

Proof:

*Sort the points $P$, and let $p_{1},\ldots,p_{n}$ he resulting order. Consider the (implicit) matrix $M=P-P$. Formally, the entry $M_{ij}$ is $p_{j}-p_{i}$ (we are interested only in the top right part of this matrix) – such an entry can be computed in $O(1)$ time directly from the sorted point set. The optimal quantity of interest is the minimum on the $k$th diagonal; that is, $\alpha(M)=\min_{i}M_{i,i+k-1}$. When a marked point get deleted, this corresponds to deleting a row and a column of $M$ – the quantity of interest remains the minimum along the $k$th diagonal. Such a deletion, as far as a specific entry of the top right of the matrix is concerned, either

(i) removes it,

(ii) keeps it in its place,

(iii) shift it one diagonal down as its moves left, or

(iv) keep it on the same diagonal as it shifts both up and left (see Figure 2.1).

In particular, any sequence of at most $q$ deletions of elements can shift an entry in the matrix at most $q$ diagonals down. This implies that we need to keep track only of the $k,\ldots,k+q$ diagonals of this matrix. To do better, observe that if we track the elements of an original diagonal of interest, the deletions can fragment the diagonal into at most $O(q)$ groups, where each group still appear as contiguous run of the original diagonal.

*To this end, let a fragment of a diagonal be either

(i) a singleton entry that appears in a row or column of a marked point, or

(ii) a maximum contiguous portion of the diagonal which does not touch any singleton entries from (i).

It is easy to verify that the $k$th diagonal of the matrix at any given point in time is made out of a sequence of at most $3k$ fragments, where each fragment is an original fragment of one of the diagonals in the range $k,\ldots,k+q$.*

As such, instead of storing all the elements of a fragment, we only maintain the minimum entry of the fragment (together with the information of what pairs of points it corresponds to). After this compression, a diagonal of interest can be represented as a linked list of $O(q)$ fragment summaries. In the preprocessing stage, the algorithm computes this representation for the $k$ to $k+q$ diagonals (using this representation). This requires $O(q^{2})$ space, and $O(nq)$ time.

A deletion of a marked point then corresponds to taking a contiguous block of linked fragments at the $i$th list and moving it to list $i-1$, doing this surgery for $i=k,\ldots,k+q$. The blocks being moved start and end in singleton entries that correspond to the deleted point. We also need to remove these two singleton elements, and merge the two adjacent fragment summaries that are no longer separated by a singleton. This surgery for all the $q+1$ lists of interest can be done in $O(q)$ time (we omit the tedious but straightforward details).

*A query corresponds to scanning the $k$th diagonal and reporting the minimum value stored along it. An unmarking operation corresponds to merging two fragment summaries and the singleton separating them into a single fragment summary, and doing this for all the $q+1$ lists. Both operations clearly can be done in $O(q)$ time.  *

Theorem 2.2**.**

*Given a set $P$ of $n$ points in the plane and an integer $k$, one can compute, in $O(n^{2}\log n)$ time, the smallest-area/perimeter axis-aligned rectangle enclosing $k$ points. *

Proof:

We do divide-and-conquer by $y$-coordinates. Given a set $P$ of $n$ points in the plane, and horizontal slabs $\sigma$ and $\tau$, each containing $q$ points of $P$, we describe a recursive algorithm to find a smallest $k$-enclosing axis-aligned rectangle containing $P$, under the restriction that the top edge is inside $\sigma$ and the bottom edge is inside $\tau$. It is assumed that either $\sigma$ is completely above $\tau$, or $\sigma=\tau$. It is also assumed that all points above $\sigma$ or below $\tau$ have already been deleted from $P$. There can still be a large number of points in $P-(\sigma\cup\tau)$ (recursion will lower $q$ but not necessarily $n$). We will not explicitly store the points in $P-(\sigma\cup\tau)$, but rather “summarize” the points in an $O(q^{2})$-space structure. Namely, we assume that the $x$-coordinates of $P$ are maintained in the 1D data structure ${\cal S}$ of Lemma 2.1, where the marked points are the $O(q)$ points in $P\cap(\sigma\cup\tau)$.

The algorithm proceeds as follows:

1. 0.

If $q=1$, then report the answer by querying ${\cal S}$ in $O(1)$ time. Else: 2. 1.

Divide $\sigma$ into two horizontal subslabs $\sigma_{1}$ and $\sigma_{2}$, each containing $q/2$ points of $P$. Likewise divide $\tau$ into $\tau_{1}$ and $\tau_{2}$. 3. 2.

For each $i,j\in\{1,2\}$, recursively solve the problem for the slabs $\sigma_{i}$ and $\tau_{j}$;222If $\sigma=\tau$, one of the four recursive calls is unnecessary. to prepare for the recursive call, make a copy of ${\cal S}$, delete the (marked) points in $P\cap(\sigma\cup\tau)$ above $\sigma_{i}$ or below $\tau_{j}$, and unmark the remaining points in $P\cap(\sigma\cup\tau)-(\sigma_{i}\cup\tau_{j})$, as shown in Figure 2.2. The time needed for these $O(q)$ deletions and unmarkings, and for copying ${\cal S}$, is $O(q^{2})$ (we emphasize that this bound is independent of $n$).

The running time satisfies the recurrence

[TABLE]

*with $T(n,1)=O(1)$, which gives $T(n,q)=O(q^{2}\log q)$. Initially, $\sigma=\tau$ is the entire plane, with $q=n$; the data structure ${\cal S}$ can be preprocessed in $O(n^{2})$ time. Thus, the total running time is $O(n^{2}\log n)$.  *

One can readily get an algorithm with $k$-sensitive running time for the perimeter case, by reducing the problem into $O(n/k)$ instance of size $O(k)$. This reduction is well known [EE94, DLSS95] in this case – approximate the smallest enclosing disk containing $k$ points in $O(n\log n)$ time, partition the plane into a grid with side length proportional to the radius of this disk, and then solve the problem for each cluster (i.e., $3\times 3$ group of grid cells) that contains at least $k$ points of $P$, using our above algorithm. We thus get the following.

Corollary 2.3**.**

*Given a set $P$ of $n$ points in the plane and an integer $k$, one can compute, in $O(n\log n+nk\log k)$ time, the smallest-perimeter axis-aligned rectangle enclosing $k$ points of $P$. *

The $O(n\log n)$ term can be eliminated in the word RAM model, using a randomized linear-time algorithm for approximate smallest $k$-enclosing disk [HM05] (which requires integer division and hashing).

A similar reduction for the minimum-area case is more challenging, and was left as an open problem in previous work [KRS17]. The difficulty arises because the optimal-area rectangle may be long and thin, with side length potentially much bigger than the radius of the minimum $k$-enclosing disk. Nonetheless, we show that such a reduction is possible (with an extra logarithmic factor) in the next subsection.

2.2 $k$-sensitive running time for smallest area

Our starting point is a shallow cutting lemma for 3-sided ranges [JL11], which we describe in detail in Appendix A for the sake of completeness. (It can be viewed as an orthogonal variant of Matoušek’s shallow cutting lemma for halfspaces [Mat92].)

Lemma 2.4** ([JL11]).**

*Given a set $P$ of $n$ points in the plane, lying above a horizontal line ${\ell}$, and a parameter $k$, one can compute a family ${\mathcal{F}}$ of at most $2\left\lceil{n/k}\right\rceil$ subsets of $P$, each of size at most $6k$. The collection of sets can be computed in $O(n)$ time if the $x$-coordinates have been pre-sorted. For any axis-aligned rectangle ${R}$ with its bottom edge lying on ${\ell}$, that contains less than $k$ points of $P$, we have $P\cap{R}\subseteq Q$ for some $Q\in{\mathcal{F}}$. *

Definition 2.5.

Let ${\mathcal{R}}$ be the set of all axis-aligned rectangles in the plane. A scoring function is a function $f:{\mathcal{R}}\rightarrow\mathbb{R}$, with the following properties:

1. (A)

Translation invariant: $\forall p\in\mathbb{R}^{2}$, we have $f(p+{R})=f({R})$. 2. (B)

Monotonicity: $\forall{R},{R}^{\prime}\in{\mathcal{R}}$, such that ${R}\subseteq{R}^{\prime}$, we have that $f({R})\leq f({R}^{\prime})$.

Functions that satisfy the above definition include area, perimeter, diameter, and enclosing radius of a rectangle.

For a set $U\subseteq\mathbb{R}^{d}$, let ${\updownarrow}{U}=\left\{(x,-y)\;\middle|\;(x,y)\in U\right\}$ be the reflection of $U$ through the $x$-axis. Similarly, let \left|{U}\right|_{y}=\left\{\bigl{.}(x,|y|)\;\middle|\;(x,y)\in U\right\} be the folding of $U$ through the $x$-axis.

Lemma 2.6**.**

*Given a set $P$ of $n$ points in the plane, a parameter $k$, and a scoring function $f$, let ${R}_{\mathrm{min}}$ be the minimum score axis-aligned rectangle that contains $k$ points of $P$, and intersects the $x$-axis. Then, one can compute a family ${\mathcal{F}}$ of $O(n/k)$ subsets of $P$, such that (i) each set of ${\mathcal{F}}$ is of size $\leq 12k$, and (ii) ${R}_{\mathrm{min}}\cap P\subseteq Q$, for some $Q\in{\mathcal{F}}$. *

Proof:

Let $P^{\prime}=\left|{P}\right|_{y}$ be the “folding” of $P$ over the $x$-axis, and let ${\mathcal{F}}^{\prime}$ be the cover of $P^{\prime}$ by sets of size $\leq 12k$, as computed by Lemma 2.4 for rectangles containing at most $2k$ points. Let ${\mathcal{F}}$ be the corresponding family of sets for $P$. We claim that ${\mathcal{F}}$ has the desired property.

Let ${R}=\left|{{R}_{\mathrm{min}}}\right|_{y}$ be the folding of ${R}_{\mathrm{min}}$, and let ${R}^{\prime}={\updownarrow}{{R}}$. Observe that $f({R})=f({R}^{\prime})\leq f({R}_{\mathrm{min}})$ because of the translation invariance of $f$, and monotonicity of $f$.

*If $|{R}\cap P|>k$ then one can shrink it so that it contains only $k$ points of $P$, but this would imply that ${R}_{\mathrm{min}}$ is not the minimum, a contradiction. The case that $|{R}^{\prime}\cap P|>k$ leads to a similar contradiction. We conclude that ${R}\cup{R}^{\prime}$ contains at most $2k$ points of $P$. Implying that ${R}=\left|{{R}_{\mathrm{min}}}\right|_{y}$ contains at most $2k$ points of $P^{\prime}$. As such, there is a set $Q^{\prime}\in{\mathcal{F}}^{\prime}$ that contains ${R}\cap P^{\prime}$. Now, let $Q$ be the corresponding set in ${\mathcal{F}}$ to $Q^{\prime}$. Since ${R}_{\mathrm{min}}\subseteq{R}\cup{R}^{\prime}$, it follows that ${R}_{\mathrm{min}}\cap P\subseteq({R}\cup{R}^{\prime})\cap P\subseteq Q$, as desired.  *

Lemma 2.7**.**

*Given a set $P$ of $n$ points in the plane, a parameter $k$, and a scoring function $f$, let ${R}_{\mathrm{min}}$ be the minimum score rectangle that contains $k$ points of $P$. One can compute, in $O(n\log n)$ time, a family ${\mathcal{F}}$ of $O(\tfrac{n}{k}\log\tfrac{n}{k})$ subsets of $P$, such that (i) each subset of ${\mathcal{F}}$ is of size $\leq 12k$, and (ii) ${R}_{\mathrm{min}}\cap P\subseteq Q$, for some $Q\in{\mathcal{F}}$. *

Proof:

*Find a horizontal line ${\ell}$ that splits $P$ evenly, and compute the family of Lemma 2.6. Now recurse on the points above ${\ell}$ and the points below ${\ell}$. The recursion bottoms out when the number of points is $\leq 12k$. The correctness is by now standard – as soon as a recursive call picks a line that stabs the optimal rectangle, the family generated for this line contains the desired set. The $x$-coordinates need to be pre-sorted just once at the beginning.  *

Theorem 2.8**.**

*Let $P$ be a set of $n$ points in the plane, $k$ be a parameter, $f$ be a scoring function for rectangles, and let alg be an algorithm the computes, in $T_{\text{{alg}}}(m)$ time, the axis-aligned rectangle containing $k$ points in a set of $m$ points that minimizes $f$. Then one can compute the rectangle containing $k$ points of $P$ that minimizes $f$, in time O\bigl{(}n\log n+T_{\text{{alg}}}(12k)\tfrac{n}{k}\log\tfrac{n}{k}\bigr{)}. *

Proof:

*Compute the family of sets ${\mathcal{F}}$ using Lemma 2.7, and then apply alg to each set in this family.  *

Combining Theorem 2.2 with Theorem 2.8 gives the following.

Corollary 2.9**.**

*Given a set $P$ of $n$ points in the plane and an integer $k$, one can compute, in $O(nk\log\tfrac{n}{k}\log k)$ time, the smallest-area axis-aligned rectangle enclosing $k$ points of $P$. *

For the case when $t=n-k$ is very small (i.e., finding the smallest enclosing axis-aligned rectangle with $t$ outliers), there is an easy reduction (for both perimeter and area) yielding $O(n+T_{\text{{alg}}}(4t))$ time [SK98, ABC*+*09], by keeping the $t$ leftmost/rightmost/topmost/bottommost points. Immediately from Theorem 2.2, we get $O(n+t^{2}\log t)$ running time.

2.3 An approximation algorithm for smallest area

In this subsection, we give an efficient approximation algorithm for the smallest-area $k$-enclosing rectangle problem. The smallest perimeter case is straightforward to approximate, by grid rounding, but the area case is tougher, again because the optimal rectangle may be long and thin.

Definition 2.10.

A laminar family of 3-sided rectangles is a collection ${\cal R}$ of axis-aligned rectangles with the bottom edges lying on the $x$-axis, such that for every pair of rectangles $[a,b]\times[0,c]$ and $[a^{\prime},b^{\prime}]\times[0,c^{\prime}]$ in ${\cal R}$, one of the following is true:

• •

$[a,b]\cap[a^{\prime},b^{\prime}]=\emptyset$, or

• •

$[a,b]\subseteq[a^{\prime},b^{\prime}]$ and $c>c^{\prime}$, or

• •

$[a^{\prime},b^{\prime}]\subseteq[a,b]$ and $c^{\prime}>c$.

Standard range trees can answer orthogonal range counting queries (counting the number of points inside rectangular ranges) in logarithmic time per query (this has been improved to $O(\sqrt{\log n})$ in the offline setting by Chan and Pătraşcu [CP10]). The following lemma shows how to achieve constant time per query in the offline laminar special case, which will be useful later in our approximation algorithm.

Lemma 2.11**.**

*Let $P$ be a set of $n$ points, and let ${\cal R}$ be a laminar family of $O(n)$ 3-sided rectangles in the plane. Suppose that we are given a designated point on the top edge of each rectangle in ${\cal R}$, and the $x$- and $y$-coordinates of all the designated points and all the points of $P$ have been pre-sorted. Then we can count, for each rectangle $R\in{\cal R}$, the number of points of $P$ inside the rectangle, in $O(n)$ total time. *

Proof:

We describe a sweep algorithm, with a horizontal sweep line $\ell$ moving downward. Let $X$ denote the set of $x$-coordinates of all points of $P$ and all designated points of the rectangles of ${\cal R}$. Consider the union of $R\cap\ell$ over all $R\in{\cal R}$; it can be expressed as a union of disjoint intervals. Let ${\cal I}_{\ell}$ be the $x$-projection of these disjoint intervals. Store the following collection $\Gamma_{\ell}$ of disjoint sets in a union-find data structure [Tar75]: for each interval $I\in{\cal I}_{\ell}$, define the set $X\cap I$, and for each $a\in X$ not covered by ${\cal I}_{\ell}$, define the singleton set $\{a\}$. Create a linked list $L_{\ell}$ containing these sets in $\Gamma_{\ell}$ ordered by $x$. For each set in $\Gamma_{\ell}$, we store a count of the number of points of $P$ below $\ell$ with $x$-coordinates inside the set.

Suppose that the sweep line $\ell$ hits the top edge of a rectangle $R$ with $x$-projection $[a,b]$. By definition of a laminar family, any interval in ${\cal I}_{\ell}$ that intersects $[a,b]$ must be contained in $[a,b]$. We find the set in $\Gamma_{\ell}$ that contains the $x$-coordinate of the designated point of $R$. From this set, we walk through the list $L_{\ell}$ in both directions to find all sets contained in $[a,b]$, and replace these sets with their union in $\Gamma_{\ell}$ and $L_{\ell}$. The count for the new set is the sum of the counts of the old sets; this also gives the output count for the rectangle $R$.

Next, suppose that the sweep line $\ell$ hits a point $p\in P$. We find the set in $\Gamma_{\ell}$ that contains the $x$-coordinate of $p$, and decrement its count. (For example, if the set is a singleton, its count changes from 1 to 0.)

*The entire sweep performs $O(n)$ union and find operations. Gabow and Tarjan [GT85] gave a linear-time union-find algorithm for the special case where the “union tree” is known in advance; their algorithm is applicable, since the union tree here is just a path of the elements ordered by $x$.  *

We first solve the approximate decision problem:

Lemma 2.12**.**

*Given a set $P$ of $n$ points in the plane, a value ${\alpha}$, and parameters $k$ and ${\varepsilon}\in(0,1)$, one can either compute a $k$-enclosing axis-aligned rectangle ${R}^{\prime}$ such that \mathrm{area}\left({{R}^{\prime}}\right)\leq\bigl{(}1+O({\varepsilon})\bigr{)}{\alpha}, or conclude that the smallest-area $k$-enclosing axis-aligned rectangle has area greater than ${\alpha}$. The running time of the algorithm is O\bigl{(}{\varepsilon}^{-3}\log{\varepsilon}^{-1}\cdot n\log n\bigr{)}. *

Proof:

It is sufficient to solve the problem for the case where the rectangle must intersect a horizontal line ${\ell}$ in $O((1/{\varepsilon})^{3}\log(1/{\varepsilon})\cdot n)$ time, assuming that the $x$- and $y$-coordinates of the given points $P$ have been pre-sorted. Then standard divide-and-conquer by $y$-coordinates gives an $O((1/{\varepsilon})^{3}\log(1/{\varepsilon})\cdot n\log n)$-time algorithm for the general problem. Pre-sorting needs to be done only once at the beginning.

Without loss of generality, assume that ${\ell}$ is the $x$-axis. Suppose there exists a rectangle ${R}^{*}$ intersecting ${\ell}$ that contains at least $k$ points and has area at most ${\alpha}$. By symmetry, we may assume that ${R}^{*}$ has greater area above ${\ell}$ than below, and that the top edge passes through some input point $p=(p_{x},p_{y})\in P$. Then the height $h^{*}$ of ${R}^{*}$ is between $p_{y}$ and $2p_{y}$, and the width $w^{*}$ is at most ${\alpha}/p_{y}$.

Without loss of generality, assume that all $x$-coordinates are in $[0,1/3]$. Define a one-dimensional quadtree interval (also known as a dyadic interval) to be an interval of the form $[\tfrac{m}{2^{i}},\tfrac{m+1}{2^{i}}]$. It is known that every interval of length $w<1/3$ is contained in a quadtree interval of length $O(w)$ after shifting the interval by one of two possible values $s\in\{0,1/3\}$ (this is a special case of a shifting lemma for $d$-dimensional quadtrees [Ber93, Cha98]). Thus, suppose that $[p_{x}-{\alpha}/p_{y},p_{x}+{\alpha}/p_{y}]$ is contained in the interval $[\tfrac{m}{2^{i}}+s,\tfrac{m+1}{2^{i}}+s]$, where the length $\tfrac{1}{2^{i}}$ is equal to the smallest power of 2 greater than $c{\alpha}/p_{y}$, for some constant $c$. (Note that $i$ is a nondecreasing function of $p_{y}$.) Without loss of generality, assume that $1/{\varepsilon}=2^{E}$ for an integer $E$. Define a family of $O(1/{\varepsilon}^{3})$ canonical rectangles of the form

[TABLE]

over all possible indices $j,j^{\prime},j^{\prime\prime}\in\{0,\ldots,1/{\varepsilon}\}$ such that $p_{x}\in[\tfrac{m}{2^{i}}+\tfrac{j}{2^{i+E}}+s,\ \tfrac{m}{2^{i}}+\tfrac{j^{\prime}}{2^{i+E}}+s]$.

By rounding, $R^{*}$ is contained in a canonical rectangle $R^{\prime}$ with height at most $h^{*}+O({\varepsilon})p_{y}\leq(1+O({\varepsilon}))h^{*}$ and width at most

[TABLE]

and thus area at most $(1+O({\varepsilon})){\alpha}$. So, it suffices to count the number of points inside each canonical rectangle and return the smallest area among those rectangles containing at least $k$ points.

*To speed up range counting, observe that for canonical rectangles with the same $j,j^{\prime},j^{\prime\prime}$, the same $s\in\{0,1/3\}$, and the same value for $(i\bmod E)$, the portion of the rectangles above (resp. below) ${\ell}$ forms a laminar family. This is because: (i) in the $x$-projections, if a pair of intervals intersects, one interval must be contained in the other; (ii) as the height of the 3-sided rectangle increases, $p_{y}$ increases, and so $i$ can only increase (or stay the same), and so the width of the rectangle can only decrease (or stay the same). Thus, we can apply Lemma 2.11 to compute the counts of the points inside each rectangle, for all canonical rectangles with a fixed $j,j^{\prime},j^{\prime\prime},s$ and $(i\bmod E)$, in $O(n)$ time (for each canonical rectangle, we can use a point $(p_{x},p_{y})\in P$ on the top edge, and a corresponding point $(p_{x},-j^{\prime\prime}{\varepsilon}p_{y})$ on the bottom edge, as the designated points). The number of choices for $j,j^{\prime},j^{\prime\prime},s$ and $(i\bmod E)$ is $O((1/{\varepsilon})^{3}\log(1/{\varepsilon}))$.  *

Theorem 2.13**.**

*Given a set $P$ of $n$ points in the plane, and parameters $k$ and ${\varepsilon}\in(0,1)$, one can compute a $k$-enclosing rectangle ${R}^{\prime}$ such that $\mathrm{area}\left({{R}^{\prime}}\right)\leq(1+{\varepsilon})\mathrm{opt}\left({P,k}\right)$, where $\mathrm{opt}\left({P,k}\right)$ is the area of the smallest axis-aligned rectangle containing $k$ points of $P$. The expected running time of the algorithm is O\bigl{(}(1/{\varepsilon})^{3}\log(1/{\varepsilon})\cdot n\log n\bigr{)}. *

Proof:

We can use known techniques for reducing optimization problems to decision problems. We give a self-contained description of one approach based on Chan’s randomized technique [Cha99].

Let $b$ be a sufficiently large constant. Divide the plane into $b$ columns (vertical slabs) each containing $n/b$ points. Similarly divide the plane into $b$ rows (horizontal slabs) each containing $n/b$ points. These steps take linear time by invoking a selection algorithm $O(b)$ times. For each quadruple $\tau=(c,c^{\prime},r,r^{\prime})$ where $c$ and $c^{\prime}$ are columns (with $c$ left of $c^{\prime}$ or $c=c^{\prime}$) and $r$ and $r^{\prime}$ are rows (with $r$ below $r^{\prime}$ or $r=r^{\prime}$), consider the subproblem of finding the smallest-area rectangle containing $k$ points of $P$, subject to the extra constraints that the left edge of the rectangle lies in $c$, the right edge lies in $c^{\prime}$, the bottom edge lies in $r$, and the top edge lies in $r^{\prime}$. To solve this subproblem, it suffices to consider the at most $4n/b$ points in $P\cap(c\cup c^{\prime}\cup r\cup r^{\prime})$. To ensure that the extra constraints are satisfied, we add $4n/b$ copies of the four intersection points formed by the right boundary of $c$, the left boundary of $c^{\prime}$, the top boundary of $r$, and the bottom boundary of $r^{\prime}$; and we add $16n/b$ to $k$. (Straightforward modifications can be made in the special case when $c=c^{\prime}$ or $r=r^{\prime}$.) Let $P_{\tau}$ be the resulting point set of size at most $20n/b$ points, and $k_{\tau}$ be the resulting value of $k$. We thus have

[TABLE]

To compute an approximation to the minimum, we consider the at most $b^{4}$ quadruples in random order $\tau_{1},\tau_{2},\ldots$ and keep track of an approximate minimum ${\alpha}$ with the invariant that ${\alpha}\leq\min\{\mathrm{opt}\left({P_{\tau_{1}},k_{\tau_{1}}}\right),\ldots,\mathrm{opt}\left({P_{\tau_{i-1}},k_{\tau_{i-1}}}\right)\}<(1+{\varepsilon}){\alpha}$ after the $(i-1)$th iteration. Let ${\varepsilon}^{\prime}$ be such that $(1+{\varepsilon}^{\prime})^{2}=1+{\varepsilon}$; note that ${\varepsilon}^{\prime}=\Theta({\varepsilon})$. At the $i$th iteration, we run the approximate decision procedure for $P_{\tau_{i}}$ twice, at values ${\alpha}$ and ${\alpha}/(1+{\varepsilon}^{\prime})$, which allows us to conclude one of the following:

• •

$\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)\geq{\alpha}$. In this case, we can continue to the next iteration and the invariant is maintained.

• •

${\alpha}/(1+{\varepsilon}^{\prime})\leq\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)<(1+{\varepsilon}^{\prime}){\alpha}$. In this case, we reset ${\alpha}$ to ${\alpha}/(1+{\varepsilon}^{\prime})$ and the invariant is maintained.

• •

$\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)<{\alpha}$. In this case, we recursively compute an approximation ${\alpha}_{i}$ to $\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)$, satisfying ${\alpha}_{i}\leq\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)<(1+{\varepsilon}){\alpha}_{i}$. We reset ${\alpha}$ to ${\alpha}_{i}$ and the invariant is maintained.

We land in the third case only if $\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)$ is the smallest among the $i$ values $\mathrm{opt}\left({P_{\tau_{1}},k_{\tau_{1}}}\right),\ldots,$ $\mathrm{opt}\left({P_{\tau_{i}},k_{\tau_{i}}}\right)$, which happens with probability at most $1/i$. Thus, the expected number of recursive calls is bounded by the $(b^{4})$th Harmonic number $\sum_{i=1}^{b^{4}}1/i<\ln(b^{4})+1$. The expected running time satisfies the recurrence

[TABLE]

*which gives T(n)=O\bigl{(}(1/{\varepsilon})^{3}\log(1/{\varepsilon})\cdot n\log n\bigr{)} when $b=1000$, for example.  *

3 Extensions

3.1 3-sided smallest $k$-enclosing rectangle

In this subsection, we give a slightly faster algorithm for the 3-sided variant of the problem, finding the smallest-area/perimeter rectangle enclosing $k$ points, under the restriction that the bottom edge lies on the $x$-axis. The improvement uses the latest result on the (min,+)-convolution problem, and is interesting in view of a reduction in Section 3.5 in the reverse direction, establishing essentially an equivalence of the 3-sided problem to (min,+)-convolution.

Problem 3.1.

(min,+)-Convolution. Given real numbers $a_{0},\ldots,a_{n-1},b_{0},\ldots,b_{n-1}$, compute $c_{\ell}=\min_{i=0}^{\ell}(a_{i}+b_{\ell-i})$ for all $\ell=0,\ldots,2n-2$.

Let $T_{\mbox{\scriptsize\rm convol}}(n)$ be the time complexity of the (min,+)-convolution problem. As observed by Bremner et al. [BCD*+*14], the problem can be reduced to (min,+)-matrix multiplication, and using the current best result by Williams [Wil14] (derandomized by Chan and Williams [CW16]), $T_{\mbox{\scriptsize\rm convol}}(n)=O(n^{2}/2^{\Omega(\sqrt{\log n})})$. We use (min,+)-convolution to speed up the preprocessing time of the 1D data structure from Section 2.1.

Lemma 3.2**.**

*The preprocessing time in Lemma 2.1 can be reduced to $O((n/q)T_{\mbox{\scriptsize\rm convol}}(q)+q^{3})$. *

Proof:

Divide the $n\times n$ matrix $M$ vertically into $n/q$ submatrices $M_{1},\ldots,M_{n/q}$ each of dimension $n\times q$. For each submatrix $M_{i}$, we consider the portions of the diagonals $k,\ldots,k+q$ that are within $M_{i}$ – each such portion will be called a chunk. We precompute the minimum of the entries in each chunk. For a fixed $i$, this is equivalent to computing $\min_{qi<j\leq q(i+1)}(p_{j+\ell-1}-p_{j})$ for all $\ell\in\{k,\ldots,k+q\}$. Notice that after some massaging of the sequence (negating, reversing, and padding), this computation can be reduced to (min,+)-convolution over $O(q)$ elements, and can thus be done in $O(T_{\mbox{\scriptsize\rm convol}}(q))$ time. The total time over all $i$ is $O((n/q)T_{\mbox{\scriptsize\rm convol}}(q))$.

*Recall that in the preprocessing algorithm in Lemma 2.1, we need to compute the minimum of each fragment in the $k,\ldots,k+q$ diagonals. Each fragment can be decomposed into some number of disjoint chunks plus $O(q)$ extra elements. Over all $O(q)$ diagonals, there are $O(q^{2})$ fragments and $O(n/q\cdot q)=O(n)$ chunks in total. Thus, we can compute the minima of all fragments in $O(q^{2}\cdot q+n/q\cdot q)=O(q^{3}+n)$ time, after the above precomputation of the minima of all chunks.  *

Theorem 3.3**.**

*Given a set $P$ of $n$ points in the plane and integer $k$, one can compute, in $O(n^{2}/2^{\Omega(\sqrt{\log n})})$ time, the smallest-area/perimeter axis-aligned rectangle enclosing $k$ points of $P$, under the restriction that the bottom edge lies on the $x$-axis. *

Proof:

Divide the plane into $n/q$ horizontal slabs each containing $q$ points, for some parameter $q$ to be set later.

Take such a slab $\sigma$. We solve the subproblem of finding a smallest $k$-enclosing axis-aligned rectangle under the restriction that the top edge is in $\sigma$ and the bottom edge is on the $x$-axis. To this end, we first delete all points above $\sigma$ or below the $x$-axis. We build the 1D data structure ${\cal S}$ in the lemma for the $x$-coordinates of the surviving points, where the marked points are the $q$ points in $\sigma$. The preprocessing time is $O((n/q)T_{\mbox{\scriptsize\rm convol}}(q)+q^{3})$. Then for each point $p\in\sigma$, we can compute a smallest $k$-enclosing axis-aligned rectangle where the top edge has $p$’s $y$-coordinate and bottom edge is on the $x$-axis, by making a copy of ${\cal S}$, deleting all points in $\sigma$ above $p$, and querying ${\cal S}$. The time needed for the $O(q)$ deletions, and for copying ${\cal S}$, is $O(q^{2})$. The total time over all $p\in\sigma$ is $O(q^{3})$.

We return the minimum (by area or perimeter) of all the rectangles found. The overall running time over all $n/q$ slabs $\sigma$ is

[TABLE]

*With $T_{\mbox{\scriptsize\rm convol}}(q)=O(q^{2}/2^{\Omega(\sqrt{\log q})})$, we can set $q=n^{1/3}$, for example, and obtain the final time bound $O(n^{2}/2^{\Omega(\sqrt{\log q})})$.  *

For $k$-sensitive bounds, we can apply the shallow cutting technique from Section 2.2 (which is easier for 3-sided rectangles) and obtain an $O(n\log n+nk/2^{\Omega(\sqrt{\log k})})$ time bound.

3.2 Arbitrarily oriented smallest $k$-enclosing rectangle

We briefly consider the problem of computing a smallest-area/perimeter arbitrarily oriented rectangle (not necessarily axis-aligned) enclosing $k$ points. The optimal rectangle is defined by 5 points, with one edge containing 2 points $p_{1}^{*}$ and $p_{2}^{*}$. Given a fixed choice of $p_{1}^{*}$ and $p_{2}^{*}$, we can use a rotation and translation to make $p_{1}^{*}p_{2}^{*}$ lie on the $x$-axis and thereby obtain a 3-sided axis-aligned rectangle problem, which can be solved in $O(n\log n+nk/2^{\Omega(\sqrt{\log k})})$ time. Exhaustively trying all pairs $p_{1}^{*}p_{2}^{*}$ then gives $O(n^{3}\log n+n^{2}k/2^{\Omega(\sqrt{\log k})})$ total time.

3.3 Minimum-weight $k$-enclosing rectangle

Our $O(n^{2}\log n)$-time algorithm can be adapted to solve the following related problem. (Without the $k$ constraint, the problem has an $O(n^{2})$-time algorithm [BCNP14].)

Theorem 3.4**.**

*Given a set $P$ of $n$ points in the plane each with a real weight, and an integer $k$, one can compute, in $O(n^{2}\log n)$ time, the axis-aligned rectangle enclosing $k$ points minimizing the total weight of the points inside. *

Proof:

We follow the same approach as in Section 2.1, with the following differences in the data structure of Lemma 2.1. For every fragment we maintain the minimum weight solution. Using prefix sums, the entry $M_{i,j}$ in the matrix contains the total weight of the elements from $i$ to $j$. As before, we break the $q+1$ diagonals of entry into fragments, where each fragment summary maintains the minimum weight encountered.

A deletion of a marked point $p$ of weight $w$ would result is an insertion of a fixup entry, of value $-w$ into a linked list of a diagonal where $p$ appeared as a singleton (when crossing a column of $p$), and a fixup entry of value $+w$ when encountering the row column of $p$. The real value of a fragment is the value stored in the fragment plus the total sum of the fixups appearing before it in the linked list of its diagonal. As such, during query the real value can be computed in $O(q)$ time overall, as this list is being scanned. When we merge two adjacent fragments separated by a singleton, we should increase the later fragment by the fixup value at the singleton before taking the minimum. Clearly, all the operations can be implemented in $O(q)$ time.

*Now, we can use the divide-and-conquer algorithm in the proof of Theorem 2.2 with no change.  *

As an application, we can solve the following problem: given $n$ points in the plane each colored red or blue, and an integer $k$, find an axis-aligned rectangle enclosing exactly $k$ points minimizing the number of red points inside. This is a special case of the problem in the above theorem, where the red points have weight 1 and blue points have weight 0, and can thus be solved in $O(n^{2}\log n)$ time.

Similarly, we can solve for other variants of the red/blue problem, for example, finding a $k$-enclosing rectangle maximizing (or minimizing) the number of red points, or finding a $k$-enclosing rectangle with exactly a given number $k_{r}$ of red points. (For the latter, the following observation allows us to reduce the 1D subproblem to querying for the maximum and minimum: given a set $P$ of red/blue points in 1D and a value $k$, let $K_{r}$ denote the set of all possible values $k_{r}$ for which there exists an interval containing $k$ points of $P$ and exactly $k_{r}$ red points; then $K_{r}$ forms a contiguous range of integers, and thus contains all numbers between $\min(K_{r})$ and $\max(K_{r})$.)

3.4 Subset sum for $k$-enclosing rectangle

A more challenging variant of the weighted problem is to find a rectangle enclosing exactly $k$ points with total weight exactly $W$ (similar to subset sum), or more generally, find an axis-aligned rectangle enclosing exactly $k$ points with total weight closest to $W$.

We use a different approach, using a 1D data structure that is static but can “plan for” a small number of deletions.

Lemma 3.5**.**

*Given a set $P$ of $n$ points in 1D and integers $k$ and $q$, we can build a static data structure, with $O(nq\log n)$ preprocessing time, that supports the following type of queries in $O(q\log n)$ time: for any subset $D\subset P$ of at most $q$ points and any weight $W$, find an interval containing $k$ points of $P-D$ with weight closest to $W$. *

Proof:

As in the proof of Lemma 2.1, we sort $P$ and consider the (implicit) matrix $M=P-P$. For each $i\in\{k,\ldots,k+q\}$, we store the elements in the $i$th diagonal in a data structure supporting 1D range predecessor/successor queries – i.e., finding the predecessor/successor to any value among the elements in any contiguous sublist of a given list – in $O(\log n)$ time, after $O(n\log n)$ preprocessing time. (See [Zho16] for the latest result on the range successor problem; for simplicity, we will ignore improvements in the logarithmic factors here.) The total preprocessing time for all $q+1$ diagonals is $O(nq\log n)$.

*To answer a query, we imagine deleting the columns and rows associated with the elements in $D$, from the matrix $M$. We want to search for $W$ in the $k$th diagonal in the modified matrix. This diagonal corresponds to $O(q)$ fragments from the $k,\ldots,k+q$ diagonals in the original matrix. Namely, as we trace the diagonal in the original matrix from left to right, whenever we hit a deleted column, we move to the diagonal one unit up, and whenever we hit a deleted row, we move the diagonal one unit down. We can search for $W$ in each fragment by a range predecessor/successor query in a diagonal in $O(\log n)$ time. We return the closest point found. The total query time is $O(q\log n)$.  *

Theorem 3.6**.**

*Given $n$ points in the plane each with a real weight, and given a real number $W$ and an integer $k$, one can compute, in $O(n^{5/2}\log n)$ time, an axis-aligned rectangle enclosing exactly $k$ points with total weight closest to $W$. *

Proof:

Divide the plane into $n/q$ horizontal slabs each containing $q$ points, for some parameter $q$ to be set later.

Take a pair of horizontal slabs $\sigma$ and $\tau$. We solve the subproblem of finding a $k$-enclosing axis-aligned rectangle with weight closest to $W$, under the restriction that the top edge is in $\sigma$ and the bottom edge is in $\tau$. To this end, we first remove all points strictly above $\sigma$ and strictly below $\tau$. We build the 1D data structure in Lemma 3.5 for (the $x$-coordinates of) the surviving points in $O(nq\log n)$ time. Then for each pair of points $p_{\sigma}\in\sigma$ and $p_{\tau}\in\tau$, we can search for a $k$-enclosing axis-aligned rectangle with weight closest to $W$, under the restriction that the top edge has $p_{\sigma}$’s $y$-coordinate and the bottom edge has $p_{\tau}$’s $y$-coordinate, by performing a query to the 1D data structure in $O(q\log n)$ time for the subset $D$ of (the $x$-coordinates of) the points above $p_{\sigma}$ in $\sigma$ and the points below $p_{\tau}$ in $\tau$. The total query time over all $O(q^{2})$ pairs of points $(p_{\sigma},p_{\tau})$ is $O(q^{3}\log n)$.

We return the closest answer found. The total running time over all $O((n/q)^{2})$ pairs of slabs $(\sigma,\tau)$ is

[TABLE]

*We set $q=\sqrt{n}$.  *

We can further improve the running time for small $k$:

Theorem 3.7**.**

*Given $n$ points in the plane each with a real weight, and given a real number $W$ and an integer $k$, one can compute, in $O(n^{2}\sqrt{k}\log k)$ time, an axis-aligned rectangle enclosing exactly $k$ points with total weight closest to $W$. *

Proof:

We first consider the variant of the problem where the rectangle is constrained to intersect a fixed vertical line $\ell$. Here, we can follow essentially the same algorithm as in Theorem 3.6 (with minor modifications to the data structure in Lemma 3.5), but when considering the subproblem for the two slabs $\sigma$ and $\tau$, we can further remove more irrelevant points: among the points strictly between $\sigma$ and $\tau$, it suffices to keep only the $k$ points immediately to the left of $\ell$ and the $k$ points immediately to the right of $\ell$. It may be too costly to compute these $O(k)$ points from scratch, but given such points for $(\sigma^{\prime},\tau)$ for the predecessor slab $\sigma^{\prime}$ of $\sigma$, we can generate the new $O(k)$ points for $(\sigma,\tau)$, by invoking a selection algorithm on $O(k+q)$ elements in $O(k+q)$ time. The number of points in the subproblem is now reduced to $O(k)$. As a consequence, the overall running time becomes

[TABLE]

We set $q=\sqrt{k}$. This gives an $O(n^{2}\sqrt{k}\log k)$-time algorithm for the restricted problem with the vertical line $\ell$.

*We can solve the original problem now via standard divide-and-conquer by $x$-coordinates, with running time given by the recurrence $T(n,k)=2\,T(n/2,k)+O(n^{2}\sqrt{k}\log k)$, which solves to T(n,k)=O\bigl{(}n^{2}\Bigl{.}\sqrt{k}\log k\bigr{)}.  *

As an application, we can solve the following problem: given $n$ colored points in the plane with $d$ different colors, and integers $k_{1},\ldots,k_{d}$, with $k_{1}+\cdots+k_{d}=k$, find an axis-aligned rectangle enclosing exactly $k_{i}$ points of the $i$th color. The problem was proposed by Barba et al. [BDF*+*13], who gave an $O(n^{2}k)$-time algorithm. (It may be viewed as a geometric variant of the jumbled or histogram indexing problem for strings [CL15].) It is a special case of the problem from Theorem 3.6: we can give points with color $i$ a weight of $M^{i}$ for a sufficiently large $M$, e.g., $M=n+1$, and set the target to $W=\sum_{i=1}^{d}k_{i}M^{i}$. Since weights require $O(d\log n)$ bits, each addition has $O(d)$ cost, and so the running time becomes $O(dn^{2}\sqrt{k}\log k)$. The weights can be reduced to $O(\log n)$ bits by randomized hashing (for example, by randomly selecting $M$ from $\{0,\ldots,p-1\}$ and working with numbers modulo $p$ for an $O(\log n)$-bit prime $p$), since there are only polynomially (i.e., $O(n^{4})$) many combinatorially different rectangles. This way, the running time can be reduced to $O(n^{2}\sqrt{k}\log k)$ – this improves Barba et al.’s result.

3.5 Conditional lower bounds

We can prove that the smallest-perimeter $k$-enclosing axis-aligned rectangle problem do not have truly subquadratic ($i.e.,O(n^{2-\delta})$) algorithms, under the conjecture that (min,+)-convolution does not have a truly subquadratic algorithm. Our proof holds for the 3-sided version of the problem, which complements nicely with our upper bound in Section 3.1 using (min,+)-convolution.

We describe a reduction from the following decision problem, which Cygan et al. [CMWW17] showed does not have a truly subquadratic algorithm under the (min,+)-convolution conjecture.

Problem 3.8.

(min,+)-Convolution Decision. Given real numbers $a_{0},\ldots,a_{n-1},b_{0},\ldots,b_{n-1},$ and $c_{0},\ldots,c_{n-1}$, decide whether

[TABLE]

Theorem 3.9**.**

*If there is a $T(n)$-time algorithm for computing the smallest-perimeter/area axis-aligned rectangle enclosing $k$ points for a given set of $n$ points in the plane and a given number $k$ (with or without the constraint that the bottom edge lies on the $x$-axis), then there is an $O(T(O(n))$-time algorithm for Problem 3.8. *

Proof:

Consider an instance of Problem 3.8. Without loss of generality, assume $a_{i},b_{j},c_{\ell}\in(0,1)$. We create an instance of the minimum-perimeter $k$-enclosing rectangle problem with $3n$ points

[TABLE]

plus $M$ extra copies of $(-a_{0},0)$ and $M$ extra copies of $(b_{0},0)$, and $k=n+2+2M$, where $M$ is a sufficiently large number, e.g., $M=2n$.

The optimal perimeter is

[TABLE]

which is at least $2n$ iff $\min_{i,j:i+j=\ell}(a_{i}+b_{j})\geq c_{\ell}$ for every $\ell$.

For minimum area, the reduction is similar, except that we replace $n-\ell-c_{\ell}$ with $\tfrac{1}{\ell+c_{\ell}}$. The optimal area is

[TABLE]

*which is at least 1 iff $\min_{i,j:i+j=\ell}(a_{i}+b_{j})\geq c_{\ell}$ for every $\ell$.  *

A similar reduction holds for the minimum-weight $k$-enclosing rectangle problem from Theorem 3.4:

Theorem 3.10**.**

*If there is a $T(n)$-time algorithm for computing the minimum-weight axis-aligned rectangle enclosing $k$ points for a given set of $n$ weighted points in the plane and number $k$ (with or without the constraint that the bottom edge lies on the $x$-axis), then there is an $O(T(O(n))$-time algorithm for Problem 3.8. *

Proof:

The reduction is similar. Assume $a_{i},b_{j},c_{\ell}\in(0,1)$. We create an instance of the minimum-weight $k$-enclosing rectangle problem with $3n$ weighted points

[TABLE]

plus $M$ extra copies of the points $(-1,0;0)$ and $(1,0;0)$, and $k=n+2+2M$, where $M$ is a sufficiently large number, e.g., $M=2n$. The third coordinate after the semicolon of each point denotes its weight. (And $a_{-1}=b_{-1}=c_{n}=0$.)

The minimum weight over all rectangles with $k$ points is equal to

[TABLE]

*which is nonnegative iff $\min_{i,j:i+j=\ell}(a_{i}+b_{j})\geq c_{\ell}$ for every $\ell$.  *

A near-quadratic conditional lower bound for the minimum-weight rectangle problem without the $k$ constraint was given by Backurs et al. [BDT16] (under a different “popular” conjecture about the complexity of maximum-weight clique).

We can similarly prove that the subset-sum variant of the $k$-enclosing rectangle problem from Theorem 3.6 (or its 3-sided variant) does not have truly subquadratic algorithms, under the conjecture that the convolution-3SUM problem (given real numbers $a_{0},\ldots,a_{n-1},b_{0},\ldots,b_{n-1},c_{0},\ldots,c_{n-1}$, decide whether $c_{\ell}=a_{i}+b_{\ell-i}$ for some $i$ and $\ell$) does not have a truly subquadratic algorithm (which is known to be true under the conjecture that 3SUM for integers does not have a truly subquadratic algorithm [Păt10]).

Appendix A Shallow cutting for points and $3$-sided rectangles

Here, we prove a shallow cutting lemma due to Jørgensen and Larsen [JL11] – we provide the full details for the sake of completeness.

Let $P$ be a set of $n$ points in general position (i.e., no two points share $x$ or $y$ values), let ${\ell}$ be a horizontal line below all the points of $P$, and let $k$ be a parameter. For simplicity of exposition we assume ${\ell}$ is the $x$-axis. We generate $m=O(n/k)$ subsets $Q_{1},\ldots,Q_{m}$ of $P$, each of size $O(k)$, such that any axis-parallel rectangle ${R}$ that has its bottom edge on ${\ell}$, and contains at most $k$ points of $P$, its associated subset of $P$ is contained in one of these subsets.

To this end, we sweep horizontally upward from ${\ell}$. At any point in time, the $x$-range is going to be split into interior disjoint intervals (the two extreme ones are rays). For an interval $I=[x_{1},x_{2}]$, its weight at time $t$ is n(I,t)=\left|{P(I,t)}\right|\bigr{.}, where $P(I,t)=P\cap\left({I\times[0,t]}\right)$ is the active set of points associated with $I$ at time $t$. As soon as the weight of some active interval $J$ becomes $2k$, say at time $t$, we stop the sweep and split $J$ into two intervals, by picking a value $m\in J$ between the $k$th rank and $(k+1)$th rank $x$-coordinates of the points of $P(I,t)$, and breaking $I$ at $m$ into two intervals. See Figure A.1. Let $I^{-},I^{+}$ be the two intervals adjacent to $I$ just before this split. We store the point $p_{t}=(m,t)$ into a set of splitting points ${S}$. Let ${R}(p_{t})=\left({I^{-}\cup I\cup I^{+}}\right)\times[0,t]$ be the rectangle associated with this split point, and add the set $P(I^{-}\cup I\cup I^{+},t)=P\cap{R}(p_{t})$ to the collection of subsets being computed.

For every two consecutive intervals $J,K$ in the final partition in the end of the sweeping, we add the set $P(I\cup J,+\infty)$ to the collection. Let ${\mathcal{F}}$ be the resulting family of sets.

Analysis.

Observe that every time we insert a point $p$ into ${S}$, we are splitting a set of $2k$ points into two sets of $k$ points, Since there are $n$ points overall, this can happen at most $n/k$ times. This readily implies that $\left|{{\mathcal{F}}}\right|\leq 2n/k$.

Lemma A.1**.**

*Let ${R}$ be any rectangle having its bottom edge on ${\ell}$, such that $\left|{{R}\cap P}\right|<k$. Then, there exists a set $Q\in{\mathcal{F}}$, such that ${R}\cap P\subseteq Q$. Furthermore, each set of ${\mathcal{F}}$ contains at most $6k$ points. *

Proof:

Let $I_{R}$ be the projection of ${R}$ to the $x$-axis. Let $I$ be the last active interval that contains $I_{R}$. If $I$ is one of the final intervals, then all the points in $I$ strip are contained in the set that corresponds to $I$ and its adjacent neighbor, and the claim immediately holds.

So, let $t$ be the critical time, where $I$ was split, and let $p$ be the splitting point. There are several possibilities.

1. I.

If $p\notin{R}$, then ${R}(p_{t})$ contains ${R}$ – indeed, ${R}(p_{t})$ $x$-axis extent contains $I$, which contains $I_{R}$. Now, ${R}\cap P\subseteq{R}(p_{t})\cap P$ is in ${\mathcal{F}}$, and the claim holds.

** 2. II.

If $p\in{R}$, then consider the lowest point ${q}$ (in the $y$-direction) of ${S}\setminus\left\{{p}\right\}$ such that its $x$-axis coordinate is in $I_{R}$.

1. II.(I)

If ${q}$ does not exist, then $I_{R}$ is contained in two consecutive final intervals, and the claim readily holds.

** 2. II.(II)

If ${q}\in{R}$, then the rectangle ${R}^{\prime}=[x(p),x({q})]\times[0,t({q})]\subseteq{R}$ contains $k$ points of $P$ (since it is one of the sides of the split created by ${q}$ – which contains exactly $k$ points by construction). But that is a contradiction to the assumption that $|{R}\cap P|<k$.

** 3. II.(III)

If ${q}\notin{R}$ then when the upward sweep line hits ${q}$, the interval $I_{R}$ is contained in two consecutive intervals, where one of them is being split by ${q}$. But then, ${R}({q})$ is taller than ${R}$, and it spans these two intervals. We conclude that ${R}\subseteq{R}({q})$, which implies the claim.

*As for the size, observe that a set in ${\mathcal{F}}$ is the union of at most three active sets, and each of these active sets contains at most $2k$ points. We conclude that a set of ${\mathcal{F}}$ contains at most $6k$ points.  *

The running time is dominated by $O(n/k)$ queries, each of the following form: report the lowest $O(k)$ points inside a given vertical slab. This is a generalization of range minimum queries, and known data structures [BFGL09] achieve $O(k)$ query time after $O(n)$ preprocessing time, assuming that the points are given in $x$-sorted order. The total construction time is $O((n/k)\cdot k)=O(n)$.

We thus get the following.

Restatement of Lemma 2.4 [JL11]. *Given a set $P$ of $n$ points in the plane, lying above a horizontal line ${\ell}$, and a parameter $k$, one can compute a family ${\mathcal{F}}$ of at most $2\left\lceil{n/k}\right\rceil$ subsets of $P$, each of size at most $6k$. The collection of sets can be computed in $O(n)$ time if the $x$-coordinates have been pre-sorted. For any axis-aligned rectangle ${R}$ with its bottom edge lying on ${\ell}$, that contains less than $k$ points of $P$, we have $P\cap{R}\subseteq Q$ for some $Q\in{\mathcal{F}}$. *