Sorted Top-k in Rounds
Mark Braverman, Jieming Mao, Yuval Peres
TL;DR
This paper analyzes the sample complexity of sorted top-$k$ problems with limited interaction rounds, providing tight bounds for noiseless and noisy comparisons, and extends results to noisy settings.
Contribution
It characterizes how the optimal sample complexity depends on the number of rounds and extends results to noisy comparisons, offering new bounds and techniques.
Findings
Sample complexity for 1 round is Θ(n^2).
Sample complexity for 2 rounds is Θ(n√k + n^{4/3}).
For r ≥ 3, sample complexity is (n^{2/r} k^{(r-1)/r} + n).
Abstract
We consider the sorted top- problem whose goal is to recover the top- items with the correct order out of items using pairwise comparisons. In many applications, multiple rounds of interaction can be costly. We restrict our attention to algorithms with a constant number of rounds and try to minimize the sample complexity, i.e. the number of comparisons. When the comparisons are noiseless, we characterize how the optimal sample complexity depends on the number of rounds (up to a polylogarithmic factor for general and up to a constant factor for or 2). In particular, the sample complexity is for , for and for . We extend our results of sorted top- to the noisy case where each comparison is correct with probability . When or 2, we…
| Number of Rounds | Upper Bound | Lower Bound |
|---|---|---|
| 1-round | ||
| 2-round | ||
| ()-round |
| Number of Rounds | Upper Bound | Lower Bound |
|---|---|---|
| 1-round | ||
| 2-round | ||
| ()-round |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComplexity and Algorithms in Graphs · Machine Learning and Algorithms · Optimization and Search Problems
\coltauthor\Name
Mark Braverman \Email [email protected]
\addrPrinceton University and \NameJieming Mao \[email protected]
\addrUniversity of Pennsylvania and \NameYuval Peres \[email protected]
Sorted Top- in Rounds
Abstract
We consider the sorted top- problem whose goal is to recover the top- items with the correct order out of items using pairwise comparisons. In many applications, multiple rounds of interaction can be costly. We restrict our attention to algorithms with a constant number of rounds and try to minimize the sample complexity, i.e. the number of comparisons.
When the comparisons are noiseless, we characterize how the optimal sample complexity depends on the number of rounds (up to a polylogarithmic factor for general and up to a constant factor for or 2). In particular, the sample complexity is for , for and for .
We extend our results of sorted top- to the noisy case where each comparison is correct with probability . When or 2, we show that the sample complexity gets an extra factor when we transition from the noiseless case to the noisy case.
We also prove new results for top- and sorting in the noisy case. We believe our techniques can be generally useful for understanding the trade-off between round complexities and sample complexities of rank aggregation problems.
keywords:
rank aggregation, sorting, top- ranking, round complexity, noisy comparisons
1 Introduction
Rank aggregation is a fundamental problem which finds numerous applications in recommendation systems, web search, social choice, peer grading and crowdsourcing. The most studied problem in rank aggregation is sorting. It aims to find the total ordering of all items. People also consider the top- problem when it is only necessary to recover the set of top- items. However, for some applications, the rank aggregation task required is neither sorting nor top-. For example, when a recommendation system shows the user a list of items, it might want to display these items in the order of recommendation. As another example, in a tournament, people usually care about the exact rankings of top players but not others.
These examples motivate us to study the sorted top- problem which lies between sorting and top-. In this problem, we have items with an underlying order and the goal is to recover the top- items with the correct order using pairwise comparisons.
In many applications, multiple rounds of interaction are costly. For example, if we collect comparison data via crowdsourcing, the comparisons can be done in parallel by different crowd workers and the total amount of time spent is mainly decided by the number of rounds. Therefore we consider the sorted top- problem in the parallel comparison model introduced by [Valiant(1975)]. In this model, an algorithm performs a set of comparisons in each round and the actual set can depend on comparison results of previous rounds. The goal is to solve the task in bounded number of rounds while minimizing the sample complexity, i.e. the total number of comparisons.
Parallel comparison algorithms have been intensively studied around 30 years ago. Much of the attention has been put into two problems: sorting [Häggkvist and Hell(1981), Ajtai et al.(1983)Ajtai, Komlós, and Szemerédi, Bollobás and Thomason(1983), Kruskal(1983), Leighton(1984), Bollobás and Hell(1985), Alon(1985), Alon et al.(1986)Alon, Azar, and Vishkin, Azar and Vishkin(1987), Pippenger(1987), Alon and Azar(1988b), Alon and Azar(1988a), Akl(1990)] and top- [Valiant(1975), Reischuk(1981), Ajtai et al.(1986)Ajtai, Komlos, Steiger, and Szemeredi, Pippenger(1987), Alon and Azar(1988b), Alon and Azar(1988a), Azar and Pippenger(1990), Bollobás and Brightwell(1990)]. While being very related to sorting and top-, sorted top- has not been studied in the parallel comparison model.
Without the constraint on the number of rounds, sorted top- can be easily solved by combining sorting and top- algorithms: we can first use a top- algorithm to find the set of top- items and then use a sorting algorithm to sort these items. If we are using sorting and top- algorithms with optimal sample complexities, one can easily show that the combination gives a sorted top- algorithm with optimal sample complexity (up to a constant factor).
However, if we only have bounded number of rounds, such combining algorithm might not give the optimal sample complexity. The adaptiveness of such combining procedure splits the rounds into rounds used by the top- algorithm and the rounds used by the sorting algorithm. As we will see later, this is not the optimal way to solve sorted top- in bounded number of rounds.
In this paper, we show optimal sample complexity bounds (up to poly logarithmic factors) of sorted top- in rounds for any constant , as shown in Table 1. Our bounds are tight up to constant factors when or .
We further extend our results to sorted top- in the noisy case where each comparison is correct with probability . This is a very simple and basic noise model. As shown in Table 2, we get tight bounds (up to a constant factor) when or 2. Compared with the sample complexity in the noiseless case, the sample complexity in the noisy case just has an extra factor when or 2.
Our techniques also give new results for top- and sorting in the noisy case. Our sorted top- algorithms are based on our top- algorithms. For top- in the noisy case, we show the tight sample complexity bounds are for and for . On the other hand, sorting is a sub-case of sorted top- by picking . Our sorted top- results imply tight bounds for sorting in the noisy case: for and for .
1.1 Related Work
Sorted top- is first discussed in [Chambers(1971)] and is referred as “partial sorting”.
The parallel comparison model is introduced by [Valiant(1975)]. Two problems which are related to sorted top- have been widely studied in this model: sorting [Häggkvist and Hell(1981), Ajtai et al.(1983)Ajtai, Komlós, and Szemerédi, Bollobás and Thomason(1983), Kruskal(1983), Leighton(1984), Bollobás and Hell(1985), Alon(1985), Alon et al.(1986)Alon, Azar, and Vishkin, Azar and Vishkin(1987), Pippenger(1987), Alon and Azar(1988b), Alon and Azar(1988a), Akl(1990)] and selection [Valiant(1975), Reischuk(1981), Ajtai et al.(1986)Ajtai, Komlos, Steiger, and Szemeredi, Pippenger(1987), Alon and Azar(1988b), Alon and Azar(1988a), Azar and Pippenger(1990), Bollobás and Brightwell(1990)]. Selection (i.e. finding the item of rank exactly ) has been shown to be very similar to top- in [Braverman et al.(2016)Braverman, Mao, and Weinberg].
The noisy comparison model was introduced by [Feige et al.(1994)Feige, Raghavan, Peleg, and Upfal]. Recently, there are several work studying top- with noisy comparisons in bounded number of rounds [Braverman et al.(2016)Braverman, Mao, and Weinberg, Agarwal et al.(2017)Agarwal, Agarwal, Assadi, and Khanna, Cohen-Addad et al.(2018)Cohen-Addad, Mallmann-Trenn, and Mathieu]. [Braverman et al.(2016)Braverman, Mao, and Weinberg] shows that the sample complexity of top- in the noisy case is when and when and . [Cohen-Addad et al.(2018)Cohen-Addad, Mallmann-Trenn, and Mathieu] gives tight sample complexity bound for general and . The sample complexity of top- in the noisy case for and 2 is not addressed in previous work.
Rank aggregation with noisy comparisons is also widely studied without the round constraint and with various noise models: [Kenyon-Mathieu and Schudy(2007), Ailon et al.(2008)Ailon, Charikar, and Newman, Braverman and Mossel(2008), Braverman and Mossel(2009), Ailon(2011), Jamieson and Nowak(2011), Lu and Boutilier(2011), Makarychev et al.(2013)Makarychev, Makarychev, and Vijayaraghavan, Wauthier et al.(2013)Wauthier, M.Jordan, and Jojic, Rajkumar and Agarwal(2014), Chen and Suh(2015), Shah and Wainwright(2015), Mohajer and Suh(2016), Negahban et al.(2017)Negahban, Oh, and Shah, Chen et al.(2017)Chen, Gopi, Mao, and Schneider, Shah et al.(2017)Shah, Balakrishnan, Guntuboyina, and Wainright, Suh et al.(2017)Suh, Tan, and Zhao, Chen et al.(2018)Chen, Li, and Mao].
2 Model and Preliminaries
We consider the sorted top- problem together with two related problems: sorting and top-. In these problems, there is a set of items with an underlying order and the goals are different:
- •
Sorted top-: output the sorted list of items with highest ranks.
- •
Sorting: output the ranks of all items.
- •
Top-: output the set of items with highest ranks.
Algorithms are allowed to make pairwise comparisons. And we want the algorithm to minimize the sample complexity, i.e. the total number of comparisons. We have two cases, with respect to comparisons: the noiseless case and the noisy case. In the noiseless case, comparisons results are always consistent with the underlying order. In the noisy case, each pairwise comparison is correct (consistent with the underlying order) with some constant probability independently. Without loss of generality, we assume each comparison is correct with probability independently.
We consider algorithms with bounded number of rounds. In each round, an algorithm needs to perform all comparisons simultaneously. We use to denote the number of rounds and we only consider cases when is a fixed constant.
We allow algorithms to use randomness. In the noiseless case, the algorithm needs to be always correct and the sample complexity is the expected total number of comparisons. In the noisy case, because of the noise, no algorithms can always be correct. The algorithm needs to be correct with probability and the sample complexity is the worst-case total number of comparisons. Notice that the requirement in the noiseless case is stronger as the always correct algorithm with expected number of comparisons can be easily made into an algorithm which succeeds with probability and worst-case number of comparisons by halting the algorithm when making too many comparisons.
3 Main Results and Proof Overviews
In this section, we show our main results and give overviews of our proof techniques.
3.1 Sorted Top- in the Noiseless Case
In this sub-section, we show our results for sorted top- in the noiseless case. All the detailed discussions and proofs can be found in Section A.
When we only have 1 round (i.e. ), it is not hard to show that comparing all pairs of items using comparisons gives optimal sample complexity (up to a constant factor). We formally discuss this in the beginning of Section A. For , we have the following two main theorems for upper and lower bounds.
Theorem 3.1**.**
For , there exists an -round algorithm that solves sorted top- with comparisons in expectation. There exists a -round algorithm that solves sorted top- with comparisons in expectation.
The main idea of the algorithm in Theorem 3.1 is to use “pivot items”. These pivot items are compared to all items. From these comparisons, we learn not only their ranks but also which items rank between two pivot items. After that, items are partitioned into chunks and we just need to solve sub problems inside chunks. See Figure 1 for a graphical view of pivot items.
Suppose we plan to use comparisons. The most naive way of using pivot items is to pick pivot items at random in the first round and compare them to all items in the same round. After this round, we will be left with chunks of items partitioned by pivot items. Since now we know the ranks of pivot items, we know which chunks have top- items and we only need to care about these chunks. We use the remaining round to run the -round sorting algorithm of [Alon et al.(1986)Alon, Azar, and Vishkin] in each such chunk in parallel. This approach with proper setting of matches the optimal sample complexity bound (up to a constant factor) if or is larger than the expected chunk size (i.e. ). It is formally described in Algorithm 2 in Section A. See also Figure 2 for a graphical view of the algorithm.
However, when and is small enough so that the first chunk is likely to have size much larger than (see Figure 3), the above approach becomes sub-optimal. At a high level, the reason is that the random pivot items chosen in the first round are not good enough to partition the top- items into small chunks. Therefore, instead of running the sorting algorithm on the first chunk in the remaining rounds, we spend one more round (round 2) to compare items to more “accurate” pivot items, partition them into smaller chunks and sort each chunk in the remaining rounds. These new pivot items are better than the random pivot items for two reasons: (i) We can spend some comparisons in the first round to choose these pivot items. So they have better structural guarantees than the random pivot items. In particular, we extend the algorithmic technique of [Braverman et al.(2016)Braverman, Mao, and Weinberg] to pick pivot items which are roughly apart. (ii) Since these pivot items are compared to other items in the second round, we can use comparison results of the random pivot items. Knowing the fact that all top- items are in the first chunk partitioned by the random pivot items, we just need to compare new pivot items to items in that chunk. This is important for getting good sample complexity. The whole process of this paragraph is formally described in Algorithm 3 in Section A. See also Figure 3 for a graphical view of the algorithm.
In Section A, we combine the above two approaches to prove Theorem 3.1. Both approaches use pivot items and then bounded-round sorting. Although the sample complexity keeps decreasing when we increase the number of rounds, we have at most 2 rounds that are different from sorting no matter how large the total number of rounds is. One may wonder why we don’t use more rounds before we apply bounded-round sorting. At a high level, the reason is that what we do before sorting is more similar to a top- algorithm and more than 3 rounds of interaction do not help much with the sample complexity for top-, e.g. [Braverman et al.(2016)Braverman, Mao, and Weinberg] shows a 3-round noiseless top- algorithm with nearly optimal sample complexity .
Theorem 3.2**.**
For , -round algorithm needs comparisons in expectation to solve sorted top-. Any -round algorithm needs comparisons in expectation to solve sorted top-.
Theorem 3.2 gives matching (up to constant or polylog factors) lower bounds compared to upper bounds in Theorem 3.1. The start point of the proof is to reduce from top- or sorting to sorted top-. Indeed, sorted top- is no easier than sorting items or finding top- items over items. However, this reduction is not good enough to give us tight lower bounds.
Let us we go back to our sorted top- algorithm in Theorem 3.1 and compare how it is different from an algorithm which is given the set of top- items and just sorts these items. The main difference is that our sorted top- algorithm spends a big fraction of comparisons in the first one or two rounds on items which are not top- items. These comparisons are not useful for sorting the top- items. Moreover, we can show that, without knowing the set of top- items, not only our algorithm but also any other algorithms will make a good amount of comparisons outside top- in the first one or two rounds. For example, it is not hard show that in expectation at most fraction of first-round comparisons are between two items in top-. The argument for the second round is more complicated.
This is the critical point of our proof. Now we know that sorted top- is no easier than sorting items with unbalanced number of comparisons in rounds (fewer comparisons in the first one or two rounds). In the rest of proof, we adapt the lower bound of bounded-round sorting (Theorem 2.1 of [Alon and Azar(1988b)]) to our unbalanced setting. For details, see Section A.2.
3.2 Warm-up: Top-1 in the Noisy Case
Now we proceed to the noisy case. First of all, one could easily adapt a noiseless algorithm into the noisy case by repeating each comparison times and use union bound in the analysis. So the interesting question here is whether the sample complexity gets an extra factor or not or something in-between, when we transition from the noiseless case to the noisy case.
In this sub-section, we show an 1-round algorithm for finding top-1 in the noisy case with comparisons. The sample complexity does not get a blow-up in the noisy case. This algorithm is simpler than and different from our 1-round algorithms for top- and sorted top- in the noisy case. We offer it here as a warm-up for the noisy case.
Without loss of generality, we assume is a power of 2. If is not a power of 2, we could add fewer than dummy items to make the total number of items a power of 2. This only increase the number of comparisons by a constant factor.
In Algorithm 1, we show our main recursive procedure of finding the top-1 item in some set of size . We will show by induction in Lemma 3.3 that it uses comparisons and succeeds with probability at least . If we run , we will get an algorithm for finding top-1 within items with probability at least in the noisy case using comparisons. Notice that although the procedure is defined recursively, no pair of items in an comparison depends on other comparisons’ results. So all the comparisons can be done in 1-round.
This recursive procedure basically partitions set into two set and of equal sizes and then find the max in each set recursively: item and item . After that it compares and some times to find the max of this two. In order to make all comparisons in 1 round, we actually compare all pairs of items for and .
In order to make this recursive procedure to succeed with probability , we want that the following three steps all succeed with probability and we take a union bound: (1) finding the max of : item (2) finding the max of : item (3) finding the max of and . As you will see in the proof, the critical point of the argument is to show that the growth in the success probability (from to ) has much less effect on the sample complexity compared with the decrease of the set size (from to ).
Lemma 3.3**.**
Let . FindMax(, , ) and its descendants use at most comparisons. FindMax(, , ) succeeds to output the top-1 in with probability at least .
Proof 3.4**.**
We know that is always a power of 2, i.e. . We prove the lemma by induction on . The base case is trivial.
Let’s assume the lemma is true for , let’s consider the case for . By induction hypothesis, we know the number of comparisons in and its descendants is at most . Same for . Therefore, the total number of comparisons used by FindMax(, , ) and its descendants is
[TABLE]
In the case that is the top-1 of , is the top-1 of and the majority of comparisons between and is consistent with their true ordering, FindMax(, , ) succeeds to output the top-1 in . By induction hypothesis, each of the first two events happens with probability at least . By Chernoff bound, the third event happens with probability at least . Therefore, by union bound, FindMax(, , ) succeeds to output the top-1 in with probability at least .
3.3 Top- in the Noisy Case
In this sub-section, we discuss top- in the noisy case. All the detailed discussions and proofs about this sub-section can be found in Section B.
As discussed in the related work, top- in the noisy case has been studied in prior work when . Nothing is known when or . On the other hand, if we go back to the noiseless case, it has been shown in prior work that the sample complexity of top- is for and for .
We show top- algorithms in the noisy case in Theorem 3.5 for or . These upper bounds are tight up to a constant factor as they even match the lower bounds in the noiseless case. In other words, for top- in 1 round or 2 rounds, the sample complexity does not get an extra factor when we go from the noiseless case to the noisy case.
Theorem 3.5**.**
For top- in the noisy case, there is an 1-round algorithm with sample complexity and a 2-round algorithm with sample complexity .
Our 1-round algorithm starts by the simple idea of comparing two items times. The majority of these comparison is consistent with the true ordering with probability . By taking a union bound later in the analysis, noisy comparisons between the same pair of two items can be considered as one noiseless comparison between them.
Since we plan to repeat each comparison times and we have only comparisons, we cannot compare all pairs of items. So we use pivot items again. We pick pivot items at random and compare them to all items times. We can partition items into chunks. For items rank before or after the chunk which contains the -th item, we can easily classify them as in the top- or outside top-. For items inside this chunk, we don’t know which ones are in top-. Since each chunk has items in expectation, the number of such items is in expectation.
How do we deal with these items? We use more random pivot items. We pick random pivot items and further partition items into chunks of size in expectation. We call these new pivot items as second-level pivot items and previous pivot items as first-level pivot items (see Figure 4). Here comes to the critical point of the argument: since second-level pivot items are only used to partition items and in the analysis we are taking union bound over events, we don’t need to repeat the comparison between each pair times. Instead, we just need to repeat each comparison times. And our total number comparisons will still be .
Finally we generalize this idea to have levels of pivot items and we can classify all items into top- or bottom-. Moreover, although these pivot items are divided into different levels, they are all chosen at random and compared to all items. So all the comparisons can be placed in a single round. The whole algorithm is formally described in Algorithm 4 in Section B. See also Figure 4 for a graphical view of the algorithm.
Now we proceed to describing our 2-round top- algorithm in the noisy case. It is the most sophisticated algorithm in this paper. We are going use comparisons.
It would be good to first understand the 2-round top- algorithm in the noiseless case with comparisons. The idea is quite simple: we pick random pivot items in the first round and partition items into chunks of size in expectation. And in the second round, we just need to focus on the chunk containing the -th item. It has size in expectation and we can just compare all pairs of items in this chunk.
Now in the noisy case, how do we still use only comparisons to find top- in 2 rounds? Repeating each comparison times does not seem to work since it reduces the number of random pivot items to and we will leave a chunk of too many items (i.e. items) to the second round.
In our 2-round algorithm, we still use random pivot items in the first round. We can only compare them to all items constant times as we only have comparisons in total. We partition items into chunks as following (see also Figure 5 for a graphical view). We first put pivot items in the order of their ranks. We get this order correctly with probability after the first round by having comparison between each pair of pivot items in parallel with other comparisons. For each item , we keep a counter and compare it to pivot items one by one. The counter is increased by one if the pivot item wins the majority of comparisons with item and decreased by one otherwise. In the end, we put the item into the chunk next to the pivot item where its counter reaches its maximum. The analysis of this process is similar to a biased random walk. Notice that item ’s counter has higher chance of increasing before it reaches its actual chunk and it has higher chance of decreasing after it reaches its actual chunk. We can show that although we may fail to put item into its actual chunk, the probability it is placed chunks away from its actual chunk can be bounded by .
After the first round, we partition items into chunks of size in expectation. As discussed above, this partition is not perfectly correct but items won’t be placed too far away from their actual chunks. If the partition is perfectly correct, then we can directly use our previous 1-round top- algorithm described above as a blackbox to deal with the chunk containing the -th item. But since the partition is not perfectly correct, we have to modify the 1-round algorithm to use in the second round of our 2-round algorithm. The whole algorithm is formally described in Algorithm 5 in Section B.
3.4 Sorted Top- in the Noisy Case
In this sub-section, we show our results for sorted top- in the noisy case. All the detailed discussions and proofs can be found in Section C. In the noisy case for round number , as described in the previous sub-section, we can adapt our noiseless algorithm into a noisy algorithm with sample complexity . For , we show tight (up to a constant factor) sample complexity bounds in Theorem 3.6.
Theorem 3.6**.**
The sample complexity of sorted top- in the noisy case is for and for .
Both of our sorted top- algorithms in Theorem 3.6 are based on our top- algorithms. Our 1-round sorted top- algorithm is relatively simple given our 1-round top- algorithm. We just compare all pairs times and also runs the 1-round algorithm for top- of Theorem 3.5 in the same round. After we make the comparisons, we learn the set of top- items. The majority of comparisons between each pair is consistent to the actual rankings with probability . Since we only focus on items, we can take the union bound to show our algorithm is correct with probability at least . For details, see Algorithm 8.
Interestingly, although sorted top- is no easier than top-, getting tight bounds of sorted top- could be easier. Our 2-round sorted top- algorithm is much simpler than our 2-round top- algorithm, and it only depends on our 1-round top- algorithm. When is not tiny (), since , we just use the sorted top- algorithm in the noiseless case (Theorem 3.1) and repeat each comparison times. When is tiny (), we partition all items into random groups; find the top-1 of each group in the first round (using the 1-round top- algorithm of Theorem 3.5) and then find the sorted top- of all these top-1’s in the second round. For details, see Algorithm 9.
Now we start to describe how we prove the matching lower bounds. We start with the 1-round lower bound. The main idea of the lower bound is to show that if an algorithm does not make enough comparisons in one round, there must exist pairs of items who have consecutive ranks and are in top-, such that they are compared fewer than times. For any one such pair of items, if we just swap their ranks, the order of items in top- changes and we can show that the chance of seeing the same comparison result would at most decrease by a factor of . As long as the number of such pairs is much larger than this factor, we can show that an 1-round algorithm with not enough comparisons outputs incorrectly with large probability. For details, see Lemma C.3.
For the 2-round lower bound, we still want to show that if an algorithm does not make enough comparisons in two rounds, there must exist enough pairs of items who have consecutive ranks and are in top-, such that they are compared fewer than times for some small constant . The proof is more complicated as the a 2-round algorithms have adaptiveness, i.e. which items are compared in the second round depend on the comparison results of the first round. The main idea of the proof is to show that bounded amount of first round comparisons won’t be helpful to figure out which items are consecutively ranked. We explain proof steps in the case when here. We divide top- items into chunks of size . We show that after the first round, in a typical chunk, constant fraction of items are compared to any items in the same chunk fewer than times. We can then show that, given the first round comparison results, there are pairs of items who could be a consecutively ranked pair with not small chance. As the algorithm has (with a small enough constant factor) comparisons, we can conclude the algorithm could miss to compare many consecutively ranked pairs times. The rest of the argument is similar to the 1-round lower bound. For details, see Lemma C.7.
4 Conclusion and Open Problems
In this paper, we characterize the optimal trade-off between the sample complexity and the round complexity of sorted top- in both the noiseless case and the noisy case. For a fixed number of rounds, our sample complexity bound is tight up to a polylogarithmic factor.
When or , we can make our sample complexity bound of sorted top- tight up to a constant factor. We extend these results to top- and sorting. These bounds also allow us to study the blow up in the sample complexity when we transition from the noiseless case to the noisy case. Interestingly, for or , this blow up is different in different rank aggregation problems: in top-, in sorted top- and in sorting.
There are mainly two obstacles to getting tighter bounds for top-, sorting and sorted top- when we have more than 2 rounds. We list them as open problems here. The first one is that we don’t have tight (up to a constant factor) sample complexity bounds even in the noiseless case.
Open Problem 4.1
Get tight (up to a constant factor) sample complexity bounds for the noiseless case when .
In particular, the first step is to consider 3-round top- in the noiseless case. [Braverman et al.(2016)Braverman, Mao, and Weinberg] shows its sample complexity is . [Bollobás and Brightwell(1990)] shows that no 3-round algorithm with comparisons can find top- correctly with probability . If we only want to succeed with constant probability (for example ), the best lower bound is the trivial one: .
Once we have a good understanding of the noiseless case, we can start to think about the noisy case for .
Open Problem 4.2
Extend our techniques for or in the noisy case to the case when we have more than 2 rounds.
In the noisy case, our 2-round bounds are very different and more complicated compared to 1-round bounds. Even if we have tight bounds in the noiseless case, getting tight bounds for more than 2 rounds could be more difficult and might require new techniques.
\acks
We would like to thank Claire Mathieu for earlier discussions of this problem.
Appendix A Sorted Top- in the Noiseless Case
In this section, we show upper and lower bounds on the sample complexity for solving sorted top- in the noiseless case.
First of all, it’s easy to observe that the sample complexity for solving sorted top- in 1 round is . For the upper bound, we just need to compare all pairs (there are of them). For the lower bound, first observe that we can wlog assume the algorithm is deterministic. Then if the algorithm uses fewer than comparisons, it misses the comparison between the best item and the second best item with positive probability and therefore the algorithm cannot even guarantee to solve top-1.
For more than 1 round, we show algorithms in Section A.1 and lower bounds in Section A.2.
A.1 Algorithms
Our algorithmic results are stated in Corollary A.1 (for 2 rounds) and Corollary A.3 (for rounds). They are based on two sub-routines: Algorithm 2 and Algorithm 3. Both of them use the sorting algorithm in [Alon et al.(1986)Alon, Azar, and Vishkin] as a blackbox (Theorem A.5).
Algorithm 2 is used when is large (). In the first round, we pick a random set of size (call them “pivot items”) and partition the entire set into chunks by comparing all items to the pivot items. In the remaining rounds, we use the sorting algorithm in [Alon et al.(1986)Alon, Azar, and Vishkin] for each chunk that has top- items. We prove Algorithm 2 works in Lemma A.6.
Algorithm 3 is used when is small. Compared with Algorithm 2, we pick the pivot items more carefully in Algorithm 3. In the first round, we extend the result of [Braverman et al.(2016)Braverman, Mao, and Weinberg] (stated in Theorem A.8 and Corollary A.9) to find pivot items. In the second round, we partition the entire set into chunks by comparing all items to the pivot items. In the remaining rounds, we use the sorting algorithm in [Alon et al.(1986)Alon, Azar, and Vishkin] for each chunk that has top- items. We prove Algorithm 3 works in Lemma A.11.
We first provide the final statements of our algorithmic results for sorted top- in the noiseless case:
Corollary A.1**.**
There exists a -round algorithm solves sorted top- with comparisons in expectation.
Proof A.2**.**
There are two cases:
- •
When , run Algorithm 2 to find the sorted top-. This takes comparisons in expectation.
- •
When , run Algorithm 2 to find the sorted top-* and then output sorted top-. This takes comparisons in expectation.*
Corollary A.3**.**
For , there exists an -round algorithm solves sorted top- with comparisons in expectation.
Proof A.4**.**
There are three cases:
- •
When , run Algorithm 2 to find the sorted top-. This takes comparisons in expectation.
- •
When : run Algorithm 3 to find the sorted top-. This takes comparisons in expectation.
- •
When , run Algorithm 3 to find the sorted top-* and then output sorted top-. This takes comparisons in expectation.*
Now we start to show two sub-routines: Algorithm 2 and Algorithm 3.
Theorem A.5** ([Alon et al.(1986)Alon, Azar, and Vishkin]).**
For any fixed , there exists an -round algorithm which sorts items with comparisons in expectation.
Lemma A.6**.**
For and , Algorithm 2 is always correct and uses comparisons in expectation.
Proof A.7**.**
The correctness of the algorithm is easy to check. To prove the lemma, it suffices to bound the expected number of comparisons used by the algorithm. In the first round, the algorithm uses comparisons. From round 2 to round , by Theorem A.5, the algorithm uses comparisons in expectation. It suffices to prove that . Notice that and ’s are random variables depending on the randomness of the algorithm.
For , define such that is in . We have
[TABLE]
For each , we will upper bound for . We start by considering (the set of items that have ranks no worse than item and are put into the same partitioned set as item ). The size of this set is exactly . For ,
[TABLE]
When , we have
[TABLE]
When , we have
[TABLE]
When , by concavity of for , we have
[TABLE]
So for , we have
[TABLE]
For , by the concavity of we have
[TABLE]
By symmetry, we can also get the same upper bound on . Therefore, for :
[TABLE]
Notice that . To sum up, we get
[TABLE]
Algorithm 5 in Appendix C.1.1 of [Braverman et al.(2016)Braverman, Mao, and Weinberg] can be easily extended to show the following theorem. In that algorithm, for their purpose, only for is explicitly computed after one round (denoted as in their pseudocode). However, it is not hard to see that ’s for all can be computed in the same way using the same set of comparisons. The only change is that the failure probability is multiplied by a factor of because of union bound.
Theorem A.8** ([Braverman et al.(2016)Braverman, Mao, and Weinberg]).**
There exists an 1-round algorithm with comparisons which outputs a list of item ’s for all such that with probability at least , , ’s rank is at most away from for some .
Using Theorem A.8, we can get the following corollary.
Corollary A.9**.**
For any , there exists an 1-round algorithm with comparisons which outputs a list of ’s for all such that with probability at least , , ’s rank is at most away from .
Proof A.10**.**
Set to be some value larger than ( is the one in the statement of Theorem A.8). It suffices to pick some . For each item, create copies. Run the 1-round algorithm of Theorem A.8 on these copies ( items). If the algorithm compares the copies of different items, we make an actual comparison between these two items. If the algorithm compares the copies of the same item, we just decide the comparison result based on some arbitrarily fixed order between the copies of the same item. Suppose the 1-round algorithm outputs . We output . It’s easy to check that with probability at least , , the difference between ’s rank and is at most
[TABLE]
Lemma A.11**.**
For and , Algorithm 3 is always correct and uses comparisons in expectation.
Proof A.12**.**
We first prove the correctness of Algorithm 3. If the algorithm declares FAIL in round 2, then all the pairs of items get compared. The algorithm definitely outputs the sorted top- correctly. Now we consider the case when the algorithm does not declare FAIL. First the set will have all the items in top- for some . Since , contains all the items in top-. We also have
[TABLE]
Since , we have . For all , we have . Therefore if the algorithm does not declare FAIL, we have ranks in . Therefore we know that all these ’s are in . We also have . This means the top- is inside the top-. Since the algorithm gets sorted top-, it can output sorted top- correctly.
Now we want to bound the expected number of comparisons of the algorithm. In the first round, both steps use comparisons. In the second round, the algorithm uses at most comparisons. In the third round, if the algorithm does not declare FAIL, each will have size at most . By Theorem A.5, the algorithm will use at most
[TABLE]
*comparisons in expectation. If the algorithm declares FAIL, the algorithm will use comparisons. We know the probability of FAIL is at most . Therefore in expectation, FAIL will cause at most comparisons.
A.2 Lower Bounds
We prove lower bounds in Lemma A.14 (for rounds) and Lemma A.16 (for 2 rounds). The main idea is to show that there are not many comparisons between top- items in the first 1 or 2 rounds and then reduce from sorting items for the remaining or rounds. Similarly as our algorithms, we use the lower bound for sorting (Theorem 2.1 of [Alon and Azar(1988b)]) as a blackbox. We state their theorem (Theorem A.13) in the format which is enough for our proof.
Theorem A.13** ([Alon and Azar(1988b)]).**
Suppose there are disjoint sets of item, denoted and . . The rank of each item in is known. The set of ranks in is known but all the orders of the items of the set are equally likely. Suppose in the first round the algorithm already makes comparisons between 2 items in and comparisons between an item in and an item in . Let . The expected number of comparisons to sort all the items in more rounds is at least
[TABLE]
for some constant . Here is defined as
[TABLE]
Lemma A.14**.**
For , -round algorithm needs comparisons in expectation to solve sorted top-.
Proof A.15**.**
Wlog we assume the algorithm gives each item a label in . As the algorithm has no information about the ordering, the list of these labels (the label of the rank-1 item, the label of the rank-2 item,…) is a uniformly random permutation of . After that, notice that an -round randomized algorithm is just a distribution over -round deterministic algorithms. To prove a lower bound on the expected number of comparisons, it suffices to only consider deterministic algorithms. And now the randomness only comes from how items are labeled. There are three cases to consider depending on the values of and :
Case 1: . If in the first round, the algorithm is using at least comparisons, then we already have the lower bound. Now assume that in the first round, the algorithm is using fewer than comparisons. In expectation, only fraction of comparisons in the first round are between two items in top- (we call them useful comparisons). Therefore in expectation, the algorithm has fewer than useful comparisons. By giving some extra useful comparisons to the algorithm’s first round, we can make sure that the algorithm always have at least useful comparisons in the first round and the expected number of useful comparisons in the first round is .
Now consider the case that we tell the algorithm the set of top- items after the first round and the algorithm just need to sort the set of items in the remaining rounds. By Theorem A.13 and the convexity of for , the number of comparisons we need to use in the last rounds is at least
[TABLE]
Case 2: . Set . If in the first round, the algorithm is using at least comparisons, then we already have the lower bound. Now assume that in the first round, the algorithm is using less than comparisons. In this case the expected number of comparisons between 2 items in top- is at most
[TABLE]
Define to be the set of items in top- that have no comparisons with any other items in top- in the first round. Define to be the set of items in top- that have no comparisons with any other items in top- in the first round. We have and . Then by Markov inequality, we have that with probability , both and .
Fix the comparison results of the first round. Now we will focus on the situation when and . It suffices to show that in this case, the algorithm needs comparisons in expectation to solve sorted top- in rounds. We remove some items from and such that , and . We also tell the algorithm the rank of each item in . Now consider the next round of the algorithm (the second round). If the algorithm uses at least comparisons in expectation, then we are done. Now assume the algorithm uses fewer than comparisons in expectation in the second round. Call comparisons involving at least one item in as useful comparisons. Since the algorithm has no information about which items in are in , the algorithm has at most useful comparisons in expectation. By giving some extra useful comparisons to the algorithm’s second round, we can make sure that the algorithm always have at least useful comparisons in the second round and the expected number of useful comparisons in the second round is .
By Theorem A.13 and the convexity of for , the number of comparisons we need to use in the last rounds is at least
[TABLE]
Case 3: . In this case, we have . We know that just to find the set of top- without round constraint we need at least comparisons. This simply gives the lower bound.
Lemma A.16**.**
Any -round algorithm needs comparisons in expectation to solve sorted top-.
Proof A.17**.**
Similarly as Lemma A.14, it suffices to prove the lower bound only for deterministic algorithms. There are two cases:
Case 1: . Notice that this case is the same as Case 1 in the proof of Lemma A.14 and that proof also works when . So from that we get a lower bound .
Case 2: . In this case we have . Notice that if an algorithm solves sorted top-, it also finds top-1. [Alon and Azar(1988b)] claims in the concluding remark that a 2-round algorithm which finds top-1 needs comparisons in expectation. This directly implies the lemma in this case.
Appendix B Top- in the Noisy Case
In this section, we show algorithms for top- in the noisy case. In particular, we show an 1-round top- algorithm with sample complexity in Section B.1 and a 2-round top- algorithm with sample complexity in Section B.2.
B.1 1-Round Top- Algorithm in the Noisy Case
In this sub-section, we show an 1-round top- algorithm with sample complexity (Lemma B.1).
Lemma B.1**.**
Algorithm 4 uses comparisons and outputs top- correctly with probability at least .
Proof B.2**.**
The number of comparisons used is at most
[TABLE]
Now we are going to show Algorithm 4 is correct with probability at least . Consider event as the intersection of the following events:
- •
: For each , either or there exists an item such that item ranks between and .
- •
: For each , either or there exists an item such that item ranks between and .
- •
: For each and each pair of items and such that and both and rank between and , whether beats in the -th iteration is consistent with the true ordering.
Assuming happens, it is easy to check by induction that the following is true, for :
- •
* is a subset of items rank between and . Items in have consecutive ranks. The -th item in is the -th item of the entire set.*
- •
* is the true rank of item in .*
- •
Items in are all in top-* and items in are all in bottom-.*
Finally notice that in the last iteration and . Therefore all items are placed in some or . Since , the algorithm outputs correctly. To sum up, so far we have that, implies the algorithm outputs correctly.
Now it suffices to show that happens with probability at least . We are going to analyze and then take a union bound.
- •
The probability that does not hold (i.e. ) is at most
[TABLE]
- •
Similarly as the previous bullet, .
- •
In the -th iteration, for each pair of items and such that and both and rank between and , there are comparisons between and . By Chernoff bound, the majority of these comparisons differs from the true ordering with probability at most
[TABLE]
By union bound,
[TABLE]
To sum up, by union bound,
[TABLE]
B.2 2-Round Top- Algorithm in the Noisy Case
In this sub-section, we show a 2-round top- algorithm with sample complexity (Corollary B.3).
For convenience we will assume . When is small, we can add dummy items as the top and bottom ones and then the problem reduces to the case when is large. This procedure only blows up the sample complexity by a constant factor.
We prove the following corollary for Algorithm 5. It’s based on Lemma B.9 (for the first round of the algorithm) and Lemma B.11 (for the second round of the algorithm).
Corollary B.3**.**
Algorithm 5 uses comparisons and outputs top- correctly with probability at least .
Proof B.4**.**
First of all, since the algorithm halts whenever it uses more than comparisons, the total number of comparisons used is always at most .
By Lemma B.9 and Lemma B.11, we have
[TABLE]
When both and hold, , and imply that the algorithm outputs correctly.
B.2.1 First Round of the 2-Round Algorithm
We first analyze the first round of the algorithm. Consider as the intersection of the following events ( happens when all of them happen):
- •
: Items in are correctly sorted.
- •
: ’s rank is between and .
- •
: ’s rank is between and .
- •
: , .
- •
: ,
- •
: and
We use to indicate the success of the first round. We show in Lemma B.9 that happens with probability at least .
Before proving Lemma B.9, we first prove Lemma B.5 and Lemma B.7 which analyze the biased random walk and are used in the proof of Lemma B.9.
Lemma B.5**.**
Suppose are correctly sorted. For each , and , and where .
Proof B.6**.**
When , the inequality. By symmetry, it suffices to prove the case when . If , . Now let’s focus on the case when . We have
[TABLE]
For each , by Chernoff bound, we know .
By Chernoff bound again, we have
[TABLE]
Finally we have
[TABLE]
Lemma B.7**.**
Suppose are correctly sorted. For each , let . For any , we have .
Proof B.8**.**
By symmetry, it suffices to prove that . Define and . We have . We will first give an upper bound on . By Lemma B.5 and Chu-Vandermonde identity, we have
[TABLE]
Since random variable , by Markov inequality, we have
[TABLE]
Lemma B.9**.**
* happens with probability at least .*
Proof B.10**.**
The randomness in the first round of the algorithm comes from three independent components:
- (i)
Randomness used to pick the random set . 2. (ii)
Randomness in the noise of comparisons in comparisons (b). These comparisons are used to sort items in . 3. (iii)
Randomness in the noise of comparisons in comparisons (a). These comparisons are used to compute .
They together decide whether happens. We are going to analyze (i) and (ii) separately and then analyze (iii) when fixing the randomness of (i) and (ii).
For (ii), the related event in is . For each pair , by Chernoff bound, the majority of comparisons between and in comparisons (b) is consistent with the true ordering with probability at least . By union bound, .
For (i), consider as the intersection of following events. They are all about the random set . Later we will use to analyze .
- •
: .
- •
: .
- •
: .
- •
: , and .
Now we analyze the probability holds (i.e. ). We have
[TABLE]
We also have
[TABLE]
For , define to be the indicator variable of whether (i.e. if and otherwise). We have that for any subset , . By generalized Chernoff bound (Theorem D.1), we have
[TABLE]
By Fact 1, we have
[TABLE]
Therefore
[TABLE]
By union bound
[TABLE]
By union bound again,
[TABLE]
For (iii), we are going to fix the randomness of (i) and (ii) and condition on . By , is not empty and set such that has the lowest rank in . By and , we know that . Consider event as the intersection of following events:
- •
: and .
- •
: , and .
- •
: , , .
- •
: , suppose , then .
Now we analyze the probability holds (i.e. ). For , by Lemma B.7, we have
[TABLE]
For , by Lemma B.7 again, we have
[TABLE]
For , by Lemma B.5, we have
[TABLE]
For , by Lemma B.5 again, we have
[TABLE]
By union bound, we have
[TABLE]
Then we have
[TABLE]
Finally we show that conditioned on , implies .
- •
* and : By , we know that and . Therefore and then and hold.*
- •
: By and , we know that and . Then .
- •
: By , we know that, , , and therefore .
- •
: By , we know that , and therefore . This means . The same argument will also give .
Therefore .
B.2.2 Second Round of the 2-Round Algorithm
Now we are going to assume holds and analyze the second round of the algorithm. Consider event to be:
- •
: , .
- •
: , .
- •
: , and .
- •
: .
We use to indicate the success of the second round. We show in Lemma B.11 that conditioned on , happens with probability at least .
Lemma B.11**.**
.
Proof B.12**.**
Assume holds. We first show that the algorithm does not halt. The first round of the algorithm uses comparisons. In the second round, by , the number of comparisons used by the algorithm is
[TABLE]
Therefore picking to be a large enough constant will make sure that the algorithm does not halt conditioned on .
Consider event to be the intersection of the following two events:
- •
: , and .
- •
: and each pair of item ranks in and , if and are compared in the -th iteration, then whether beats in the -th iteration of the second round is consistent with the true ordering.
Now let’s analyze . We start with . By , we know that , we have . Therefore,
[TABLE]
In the -th iteration, for each pair of item ranks in and , if and are compared in the -th iteration, then there are comparisons between and . By Chernoff bound, the majority of these comparisons differs from the true ordering with probability at most
[TABLE]
By union bound,
[TABLE]
By union bound again,
[TABLE]
Now we are going to show that and imply . Define to be the event that , to be the event that and to be the event that and . Now assume and hold.
- •
*, : By , we know that all the items in are compared to all the items in in the -th iteration. Then by we know that, , is ’s correct rank in . By , we know that either or we have does not exist and . In both cases, we have and . Similarly, we get and . So we get *and .
- •
: By , we know that . Then simply implies .
Since holds, by induction we know that , and hold. Also notice that and thus . Since implies is ’s correct rank in for , we know that and will rank at and in . And therefore we have (event ).
To sum up, we have
[TABLE]
Appendix C Sorted Top- in the Noisy Case
In this section, we consider sorted top- in the noisy case. In particular, we show that the sample complexity of 1-round algorithms is (in Section C.1) and the sample complexity of 2-round algorithms is (in Section C.2).
When , sorted top- is equivalent to top-. In this section, we only consider cases when .
C.1 1-Round Sorted Top- in the Noisy Case
In this sub-section, we show an 1-round algorithm in Lemma C.1 and a matching lower bound in Lemma C.3.
Lemma C.1**.**
We have an 1-round algorithm (Algorithm 8) which solves sorted top- in the noisy case with comparisons.
Proof C.2**.**
First of all, by the definition of Algorithm 8 and Lemma B.1 (which bounds the number of comparisons of Algorithm 4), we know Algorithm 8 uses comparisons.
By Lemma B.1, we know that the probability that Algorithm 4 outputs top- correctly with probability at least . Therefore is the set of top- items with probability at least .
For each pair of items , the probability that whether beats is consistent with their underlying order is at least . For a fixed , by union bound,
[TABLE]
By union bound again, Algorithm 8 is correct with probability at least .
Lemma C.3**.**
Any 1-round algorithm needs comparisons to output sorted top- correctly with probability at least in the noisy case.
Proof C.4**.**
For , the lemma is directly implied by 1-round lower bound of top- in the noisy case. In the rest of the proof, we assume .
Consider any algorithm with fewer than comparisons. We are going to show outputs sorted top- incorrectly with probability .
Let’s assume labels items as . Let be the random variable of the actual rank of item with label , . Before the algorithm makes any comparisons, distributed as a uniform distribution over all permutations of . Wlog we can assume is deterministic. We use to denote the random variable of the comparison results and to denote the realized value of the comparison results. We use to denote the sorted top- outputted by algorithm given comparison results .
For any ranking , define as the following:
- •
* if .*
- •
.
- •
.
For and to be any permutation of , define to be the number of comparisons between items with label and . Define . We know that for any , . Therefore, by Markov’s inequality, we know that with probability at least , . When , we know that . Therefore .
When and there are comparisons between items with labels and in algorithm , we have
[TABLE]
To sum up, we have
[TABLE]
Therefore .
C.2 2-Round Sorted Top- in the Noisy Case
In this sub-section, we show a 2-round algorithm in Lemma C.5 and a matching lower bound in Lemma C.7.
Lemma C.5**.**
We have a 2-round algorithm which solves sorted top- in the noisy case with comparisons.
Proof C.6**.**
When , we have . We just simply use the 2-round sorted top- algorithm (Algorithm 2) in the noiseless case and turn it into a 2-round sorted top- algorithm in the noisy case by repeating each comparison times. The algorithm uses comparisons.
When , we use Algorithm 9. It’s clear that this algorithm uses comparisons. We are going to show that this algorithm succeeds with probability . Consider the following events:
- •
No two items in top-* are placed in the same . This event happens with probability at least .*
- •
For each that contains a top-* item, is actually the top-1 of . This event happens with probability .*
- •
In the second round, the output is the correct sorted top-* of . This happens with probability at least .*
When all these events happen, it’s easy to check that Algorithm 9 outputs correctly. By union bound, all of these events happen with probability at least .
Lemma C.7**.**
Any 2-round algorithm needs comparisons to output sorted top- correctly with probability at least in the noisy case.
Proof C.8**.**
For notation convenience, we will wlog assume is even. For odd ’s, because sorted top- is an easier task than sorted top-, we will just apply the lower bound of sorted top-.
Consider some algorithm that uses fewer than comparisons for . We are going to show that outputs sorted top- incorrectly with probability . Let’s assume labels items as . Let be the actual ranking of item with label , . Before the algorithm makes any comparisons, distributed as a uniform distribution over all permutations of . Wlog we can assume is deterministic. We use to denote the random variable of the comparison results in the first round and to denote the random variable of the comparison results in the second round. We use to denote the full history.
Define to be the event that there exists pairs of items such that , , and are compared fewer than times in . We will first prove , and then we will show that this implies outputs incorrectly with probability at least .
We first prove . We consider two different cases depending on how large is.
- •
Case 1: . Set . Define to be the set of items ranked in for . ’s are random variables depending on . Let be the collection of and (these are labels of top-* items with odd rankings).*
For , define to be the set of items in which satisfy the following:
- –
Have even rankings.
- –
Are compared to at most items in in the first round of .
Notice that once is fixed, all ’s are fixed.
For , define to be the set of pairs of items such that
- –
* are compared more than times in the second round of .*
- –
.
- –
* has an even ranking and has an odd ranking.*
Notice that once and are fixed, ’s are fixed.
Finally for , define be the number of pairs such that
- –
.
- –
* are compared at most times in the two rounds of .*
- –
There exists such that and .
Clearly, implies , in other words, . So it suffices to prove that . Also notice that, once is fixed, ’s are independent.
Define to be the event in which there are at least ’s with . Notice that once is fixed, is also fixed. We want to show that . First of all, each comparison in the first round is a comparison between two items in the same with probability at most . So with probability at least , the number of such comparisons is at most , as there are at most comparisons in the first round. In this case, consider the top-* items with even rankings. Since , we know that at most of them are compared to items in the same more than times. Therefore, at most ’s can have . To sum up, we have .*
Now consider ’s. We know that there are at most comparisons in the second round, therefore we have . Since , at most ’s can have .
Now we fix and such that happens. We know that there are at least ’s satisfying both and . Consider any such . Define to be the set of items such that for each ,
- –
.
- –
There are at most ’s such that .
By averaging argument, we have . Now we throw away arbitrary items in to make it has size exactly .
For each item , define to be 1 if , otherwise is 0. Here is just the item ranks right above in . We have . Since we have already fix , so the items with odd rankings in have fixed rankings. Therefore only depends on .
Now consider any subset and any . Define . We fix for all in . Now consider the probability such that . For any such that , consider such that , and . We have at least many such . Define a different ranking such that it is the same as except the rankings of and are swapped. Notice that with the same , switching from to does not change and for all in , but changes from 0 to 1. We know that each of and is compared to items in at most times in the first round of , we have
[TABLE]
As , we have
[TABLE]
We know that each corresponds to at least such and each corresponds to at most such . Therefore,
[TABLE]
So we have
[TABLE]
And this implies
[TABLE]
For any ,
[TABLE]
Then we have
[TABLE]
To sum up, we have
[TABLE]
- •
Case 2: . We are going to use a similar proof strategy as Case 1 (with slight changes).
Define to be the set of top-* items and define to be the set of items in such that for each , is compared to at most items in in the first round of . Notice that once is fixed, is also fixed.*
Define to be the set of pair such that
- –
* are compared more than times in the second round of .*
- –
.
* is fixed once and are fixed. Finally for , define to be 1 if the following is true (otherwise ):*
- –
Let and .
- –
.
- –
.
Clearly, implies . In other words, . It suffices to prove .
Define to be the event that . First of all, each comparison in the first round is a comparison between two items in with probability at most . So with probability at most . the number of such comparisons is at most as there are at most comparisons in the first round of . In this case, consider items in . Since , at most are compared to items in more than times. Therefore . To sum up, we have .
Now consider . We know that there are at most comparisons in the second round of . We have .
Now we fix and such that happens. Consider any sets . Let be the largest item in and let . We also fix for all . Now consider the probability such that . For any such that , consider and such that the followings are satisfied
- –
**
- –
.
- –
, .
- –
.
We have at least such pairs.
Define a different ranking such that , and the relative positions of other items in is the same as in . Notice that with the same , switching from to does not change and , but changes from 0 to 1. We know that each of and is compared to items in at most times in the first round of , we have
[TABLE]
As , we have
[TABLE]
We know that each corresponds to at least such and each corresponds to at most such . Therefore,
[TABLE]
So we have
[TABLE]
And this implies
[TABLE]
For any ,
[TABLE]
Then we have
[TABLE]
To sum up, we have
[TABLE]
Now we have . Define to be the output of the algorithm given history . For a ranking and two labels , define as the ranking which is the same as except the positions of and are swapped. We know that if and , the sorted top- of is different from the sorted top- of . For some and , if happens, define as the set of pairs of items such that , , and are compared fewer than times in .
Now we have
[TABLE]
Therefore, we have
[TABLE]
Finally we get
[TABLE]
Appendix D Generalized Chernoff Bound
In the proofs, we use the generalized Chernoff bound of [Panconesi and Srinivasan(1997)] as stated below.
Theorem D.1** ([Panconesi and Srinivasan(1997)]).**
Let be Boolean random variables such that, for some , we have that, for every subset , , then for any , . Here is the Kullback-Leibler divergence defined below.
Definition D.2** (Kullback-Leibler Divergence).**
The Kullback-Leibler divergence for .
We have the following fact which is used to approximate the Kullback-Leibler divergence.
Fact 1**.**
For ,
[TABLE]
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[Agarwal et al.(2017)Agarwal, Agarwal, Assadi, and Khanna] Arpit Agarwal, Shivani Agarwal, Sepehr Assadi, and Sanjeev Khanna. Learning with limited rounds of adaptivity: Coin tossing, multi-armed bandits, and ranking from pairwise comparisons. In Proceedings of the 30th Conference on Learning Theory, COLT 2017, Amsterdam, The Netherlands, 7-10 July 2017 , pages 39–75, 2017.
- 2[Ailon(2011)] N. Ailon. Active learning ranking from pairwise preferences with almost optimal query complexity. In Advances in Neural Information Processing Systems , 2011.
- 3[Ailon et al.(2008)Ailon, Charikar, and Newman] N Ailon, M. Charikar, and A. Newman. Aggregating inconsistent information: ranking and clustering. Journal of the ACM , 55(5):23:1–23:27, 2008.
- 4[Ajtai et al.(1986)Ajtai, Komlos, Steiger, and Szemeredi] M Ajtai, J Komlos, W L Steiger, and E Szemeredi. Deterministic selection in o(loglog n) parallel time. In Proceedings of the Eighteenth Annual ACM Symposium on Theory of Computing , STOC ’86, pages 188–195, New York, NY, USA, 1986. ACM. ISBN 0-89791-193-8. 10.1145/12130.12149 . URL http://doi.acm.org/10.1145/12130.12149 . · doi ↗
- 5[Ajtai et al.(1983)Ajtai, Komlós, and Szemerédi] Miklós Ajtai, János Komlós, and Endre Szemerédi. An o(n log n) sorting network. In Proceedings of the 15th Annual ACM Symposium on Theory of Computing, 25-27 April, 1983, Boston, Massachusetts, USA , pages 1–9, 1983. 10.1145/800061.808726 . URL http://doi.acm.org/10.1145/800061.808726 . · doi ↗
- 6[Akl(1990)] Selim G. Akl. Parallel Sorting Algorithms . Academic Press, Inc., Orlando, FL, USA, 1990. ISBN 0120476800.
- 7[Alon(1985)] Noga Alon. Expanders, sorting in rounds and superconcentrators of limited depth. In Proceedings of the 17th Annual ACM Symposium on Theory of Computing, May 6-8, 1985, Providence, Rhode Island, USA , pages 98–102, 1985. 10.1145/22145.22156 . URL http://doi.acm.org/10.1145/22145.22156 . · doi ↗
- 8[Alon and Azar(1988 a)] Noga Alon and Yossi Azar. Sorting, approximate sorting, and searching in rounds. SIAM J. Discrete Math. , 1(3):269–280, 1988 a. 10.1137/0401028 . URL http://dx.doi.org/10.1137/0401028 . · doi ↗