Unlabeled sample compression schemes and corner peelings for ample and maximum classes
J\'er\'emie Chalopin, Victor Chepoi, Shay Moran, Manfred K., Warmuth
TL;DR
This paper explores the relationship between geometric and combinatorial concepts in machine learning, revealing errors in previous constructions and proposing a new compression scheme for maximum classes, with open questions for broader classes.
Contribution
It introduces a geometric construction that disproves prior assumptions about corners in maximum classes and presents a new unlabeled sample compression scheme.
Findings
A maximum class of VC dimension 3 has no corners, contradicting previous beliefs.
Previous optimal compression schemes for maximum classes are incorrect.
A new compression scheme for maximum classes is proposed.
Abstract
We examine connections between combinatorial notions that arise in machine learning and topological notions in cubical/simplicial geometry. These connections enable to export results from geometry to machine learning. Our first main result is based on a geometric construction by Tracy Hall (2004) of a partial shelling of the cross-polytope which can not be extended. We use it to derive a maximum class of VC dimension 3 that has no corners. This refutes several previous works in machine learning from the past 11 years. In particular, it implies that all previous constructions of optimal unlabeled sample compression schemes for maximum classes are erroneous. On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes. We leave as open whether our unlabeled sample compression scheme extends to ample (a.k.a. lopsided or extremal)…
Click any figure to enlarge with its caption.
Figure 1
Figure 2Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Unlabeled sample compression schemes and corner peelings for
ample and maximum classes∗
Jérémie Chalopin
CNRS, Aix-Marseille Université, Université de Toulon, LIS, Marseille, France
,
Victor Chepoi
Aix-Marseille Université, CNRS, Université de Toulon, LIS, Marseille, France
,
Shay Moran
Department of Mathematics, Technion and Google Research
and
Manfred K. Warmuth
Visiting Google Brain, Mountain View, California, USA. Formerly Computer Science Department, University of California, Santa Cruz, USA
Abstract.
We examine connections between combinatorial notions that arise in machine learning and topological notions in cubical/simplicial geometry. These connections enable to export results from geometry to machine learning. Our first main result is based on a geometric construction by Tracy Hall (2004) of a partial shelling of the cross-polytope which can not be extended. We use it to derive a maximum class of VC dimension 3 that has no corners. This refutes several previous works in machine learning from the past 11 years. In particular, it implies that the previous constructions of optimal unlabeled sample compression schemes for maximum classes are erroneous.
On the positive side we present a new construction of an optimal unlabeled sample compression scheme for maximum classes. We leave as open whether our unlabeled sample compression scheme extends to ample (a.k.a. lopsided or extremal) classes, which represent a natural and far-reaching generalization of maximum classes. Towards resolving this question, we provide a geometric characterization in terms of unique sink orientations of the 1-skeletons of associated cubical complexes.
Key words and phrases:
VC-dimension, Sample Compression, Sauer-Shelah-Perles Lemma, Sandwich Lemma, Maximum Class, Ample Class, Extremal Class, Corner Peeling, Unique Sink Orientation
∗An extended abstract [11] of this paper has appeared in the proceedings of ICALP 2019.
1. Introduction
The Sauer-Shelah-Perles Lemma [38, 40, 43] is arguably the most basic fact in VC theory; it asserts that any class satisfies , where . A beautiful generalization of Sauer-Shelah-Perles’s inequality asserts that , where is the family of subsets that are shattered by .111Note that this inequality indeed implies the Sauer-Shelah-Perles Lemma, since . The latter inequality is a part of the Sandwich Lemma [3, 8, 13, 33], which also provides a lower bound for (and thus “sandwiches” ) in terms of the number of its strongly shattered subsets (see Section 2). A class is called maximum/ample if the Sauer-Shelah-Perles/Sandwich upper bounds are tight (respectively). Every maximum class is ample, but not vice versa.
Maximum classes were studied mostly in discrete geometry and machine learning, e.g. [45, 18, 15, 16, 23]. The history of ample classes is more interesting as they were discovered independently by several works in disparate contexts [3, 24, 8, 29, 5, 13, 46]. Consequently, they received different names such as lopsided classes [24], extremal classes [8, 29], and ample classes [5, 13]. Lawrence [24] was the first to define them for the investigation of the possible sign patterns realized by points of a convex set of . Interestingly, Lawrence’s definition of these classes does not use the notion of shattering nor the Sandwich Lemma. In this context, these classes were discovered by Bollobás and Radcliffe [8] and Bandelt et al. [5], and the equivalence between the two definitions appears in [5]. Ample classes admit a multitude of combinatorial and geometric characterizations [5, 6, 8, 24] and comprise many natural examples arising from discrete geometry, combinatorics, graph theory, and geometry of groups [5, 24].
1.1. Main Results
1.1.1. Corner Peelings.
A corner in an ample class is any concept that belongs to a unique maximal cube of (equivalently, is a corner if is also ample, see Lemma 4.1). A sequence of corner removals leading to a single concept is called a corner peeling; corner peeling is a strong version of collapsibility. Wiedemann [46] and independently Chepoi (unpublished, 1996) asked whether every ample class has a corner. The machine learning community studied this question independently in the context of sample compression schemes for maximum classes: Rubinstein and Rubinstein [35] showed that corner peelings lead to optimal unlabeled sample compression schemes (USCS).
In Theorem 4.5 we refute this conjecture. The crux of the proof is an equivalence between corner peelings and partial shellings of the cross-polytope. This equivalence translates the question whether corners always exist to the question whether partial shellings can always be extended. The latter was an open question in Ziegler’s book on polytopes [49], and was resolved in Tracy Hall’s PhD thesis [20] where an interesting counterexample is presented. The ample class resulting from Hall’s construction yields a maximum class without corners.
1.1.2. Sample Compression.
Sample compression is a powerful technique to derive generalization bounds in statistical learning. Littlestone and Warmuth [25] introduced it and asked if every class of VC-dimension has a sample compression scheme of a finite size. This question was later relaxed by Floyd and Warmuth [16, 44] to whether a sample compression scheme of size exists. The first question was recently resolved by [31] who exhibited an sample compression scheme. The second question however remains one of the oldest open problems in machine learning (for more background we refer the reader to [30] and the books [39, 47]).
Rubinstein and Rubinstein [35, Theorem 16] showed that the existence of a corner peeling for a maximum class implies a representation map for (see Section 3 for a definition), which is known to yield an optimal unlabeled sample compression scheme of size [23].222Pálvölgyi and Tardos [34] recently exhibited a (non-ample) class with no USCS of size . They claim, using an interesting topological approach, that maximum classes admit corner peelings. Unfortunately, our Theorem 4.5 shows that this does not hold.
While our Theorem 4.5 rules out the program of deriving representation maps from corner peelings, in Theorem 5.1 we provide an alternative derivation of representation maps for maximum classes and therefore also of unlabeled sample compression schemes for them.
1.1.3. Sample Compression and Unique Sink Orientations.
We next turn to construction of representation maps for ample classes. In Theorem 6.8 we present geometric characterizations of such maps via unique sink orientations: an orientation of the edges of a cube is a unique sink orientation (USO) if any subcube has a unique sink. Szabó and Welzl [41] showed that any USO of leads to a representation map for . We extend this bijection to ample classes by proving that representation maps are equivalent to orientations of such that (i) is a USO on each subcube , and (ii) for each the edges outgoing from belong to a subcube . We further show that any ample class admits orientations satisfying each one of those conditions. However, the question whether all ample classes admit representation maps remains open.
1.1.4. Implications on Previous Works.
Our Theorem 4.5 establishes the existence of maximum classes without any corners, thus countering several previous results in machine learning:
- •
Rubinstein and Rubinstein [35, Theorem 32] showed that any maximum class can be represented by a simple arrangement of piecewise-linear hyperplanes. In [35, Theorem 39], they claim that sweeping such an arrangement leads to a corner peeling of the corresponding maximum class. This is unfortunately false, as witnessed by Theorem 4.5.
- •
Kuzmin and Warmuth [23] constructed unlabeled sample compression schemes for maximum classes based on the presumed uniqueness of a certain matching (their Theorem 10). This theorem is wrong (as explained in Section 4.2) as it implies the existence of corners and Hall’s counterexample does not have corners. However their conclusion is correct: In our Theorem 5.1 we show that such unlabeled compression schemes always exist based on a different construction and proof method.
- •
Theorem 3 by Samei, Yang, and Zilles [37] is built on a generalization of Theorem 10 from [23] to the multiclass case which is also incorrect.
- •
Theorem 26 by Doliwa et al. [12] uses the result in [35] to show that the Recursive Teaching Dimension (RTD) of maximum classes equals to their VC dimension. However the VC dimension maximum class from Theorem 4.5 has RTD at least . It remains open whether the RTD of every maximum class is bounded by .
1.1.5. An Optimal Proper PAC Learner for Maximum Classes.
In a recent work, Bousquet, Hanneke, Moran, and Zhivotovskiy [9] showed that a special type of sample compression schemes, termed stable compression schemes, achieve the optimal learning rate in PAC learning. They further noticed that any sample compression scheme which is defined by a representation map is stable. Thus, using the compression scheme constructed in this paper, Bousquet et al. conclude that every maximum class can be properly learned by an algorithm achieving the optimal learning rate.
1.2. Organization.
Section 2 presents the main definitions and notations. Section 3 reviews characterizations of ample/maximum classes and characteristic examples. Section 4 demonstrates the existence of the maximum class without corners. Section 5 establishes the existence of representation maps for maximum classes. Section 6 establishes a bijection between representation maps and unique sink orientations for ample classes.
2. Preliminaries
A concept class is a set of subsets (concepts) of a finite ground set which is called the domain of and denoted . We sometimes treat the concepts as characteristic functions rather than subsets. The support (or dimension set) of is the set .
Let be a concept class of . The complement of is . The twisting of with respect to is the concept class . The restriction of a concept on is the concept . The restriction of on is the class whose domain is . We use as shorthand for ; in particular, we write for (see Figure 1 for an example), and for for (note that ). A concept class is a cube if there exists such that and contains a single concept (denoted by ). Note that and therefore we say that is a -cube; is called the dimension of . Two cubes with the same support are called parallel cubes. A cube is maximal if there is no cube such that .
Let denote the -dimensional cube where ; are called adjacent if the symmetric difference is of size . The 1-inclusion graph of is the subgraph of induced by the vertex-set when the concepts of are identified with the corresponding vertices of . Any cube is called a cube of . The cube complex of is the set . The cubes of are called the faces of and the maximal cubes of are called the facets of . The dimension of is the largest dimension of a cube of . A concept is called a corner of if belongs to a unique maximal cube of .
The reduction of a concept class to is a concept class on which has one concept for each -cube of : . When we denote by and call it the -hyperplane of (see Figure 1 for an example). Note that a concept belongs to if and only if and both belong to . The union of all cubes of having in their support is called the carrier of and is denoted by . If , we also denote by (note that ).
The tail of a concept class on dimension consists of all concepts that do not have in an incident edge labeled with . They correspond to the concepts of , i.e., to the concepts of that have a unique extension in . The class can be partitioned as , where denotes the disjoint union and consists of all concepts in extended with bit in dimension .
Given two classes and where and are disjoint, the Cartesian product is the concept class .
A concept class is connected if the graph is connected. If is connected, denote by the graph-distance between and in and call it the intrinsic distance between and . The distance between two vertices of coincides with the Hamming distance between the 0-1-vectors corresponding to and . Let be the interval between and in ; equivalently, is the smallest cube of containing and . A connected concept class is called isometric if for any and locally isometric if for any such that . Any path of connecting two concepts and of can be lifted to a path of parallel -cubes connecting and in ; such a path of cubes is called a gallery.
A simplicial complex on a set is a family of subsets of , called simplices or faces of , such that if and , then . The facets of are the maximal (by inclusion) faces of . The dimension of is the size of its largest face. A simplicial complex is a pure simplicial complex of dimension if all its facets have size .
A set is shattered by a concept class if . Furthermore, is strongly shattered by if contains a -cube. Denote by and the simplicial complexes consisting respectively of all shattered and of all strongly shattered sets of . Clearly, and both and are closed by taking subsets, i.e., and are simplicial complexes. The classical VC-dimension [43] of a concept class is the size of the largest set shattered by , i.e., the dimension of the simplicial complex .333Note that the usual definition of the dimension of a simplicial complex is the size of the largest face of minus 1. We adopted this convention to have an equality between the VC-dimension of a class and the dimension of . The fundamental sandwich lemma (rediscovered independently by Pajor [33], Bollobás and Radcliffe [8], Dress [13], and Anstee et al. [3]) asserts that . If and , then cannot contain more than simplices, yielding the well-known Sauer-Shelah-Perles lemma [38, 40, 43] that .
A labeled sample is a set , where and . An unlabeled sample is a set , where . A subsample of a sample (labeled or unlabeled) is a subset of . Given a labeled sample , the unlabeled sample is the domain of and is denoted by . A labeled sample is realizable by a concept (seen as a map) if for every , and is realizable by a concept class if it is realizable by some . For a concept class , let be the set of all labeled samples realizable by .
A sample compression scheme for a concept class is best viewed as a protocol between a compressor and a reconstructor (which both depend of ). The compressor gets a labeled sample realizable by from which it picks a small subsample . The compressor sends to the reconstructor. Based on , the reconstructor outputs a concept that needs to be consistent with the entire input sample . A sample compression scheme has size if for every realizable input sample the size of the compressed subsample is at most . An unlabeled sample compression scheme is a sample compression scheme in which the compressed subsample is unlabeled. So, the compressor removes the labels before sending the subsample to the reconstructor. An unlabeled sample compression scheme of size for a concept class is thus defined by a (compressor) function and a (reconstructor) function such that for any realizable sample of , the following conditions hold: and .
In the literature, one usually allows the reconstructor to take values in , i.e., the reconstructor can return a subset that is not a concept of . The unlabeled sample compression schemes we consider in this paper, i.e., satisfying the property that , are usually called proper unlabeled sample compression schemes.
3. Ample and Maximum Classes
In this section, we briefly review the main characterizations and the basic examples of ample classes (maximum classes being one of them).
3.1. Characterizations
A concept class is called ample if . Ample classes are closed by taking restrictions, reductions, intersections with cubes, twistings, complements, and Cartesian products.
The following theorem reviews the main combinatorial characterizations of ample classes:
Theorem 3.1** ([5, 8, 24]).**
The following conditions are equivalent for a class :
- (1)
* is ample;* 2. (2)
* is ample;* 3. (3)
; 4. (4)
; 5. (5)
; 6. (6)
* is ample for any cube ;* 7. (7)
* for all partitions ;* 8. (8)
for all partitions , either or .444This is the original definition of lopsidedness by Lawrence [24].
Condition (3) leads to a simple definition of ampleness: is ample if whenever is shattered by , then there is a -cube in . Thus, if is ample we will write instead of . It follows that for ample classes, the VC-dimension of a concept class , the dimension of the simplicial complex , and the dimension of the cube complex of are the same. In the following, we talk about a -dimensional class when these three dimensions are equal to .
We continue with metric and recursive characterizations of ample classes:
Theorem 3.2** ([5]).**
The following are equivalent for a concept class :
- (1)
* is ample;* 2. (2)
* is connected for all ;* 3. (3)
* is isometric for all ;* 4. (4)
* is isometric, and both and are ample for all ;* 5. (5)
* is connected and all hyperplanes are ample.*
Corollary 3.3**.**
Two maximal cubes of an ample class have different supports.
Proof.
Indeed, if and are two -cubes with the same support, by Theorem 3.2(2) and can be connected in by a gallery, and thus is contained in a -cube. Therefore, and cannot be maximal. ∎
Lemma 3.4**.**
Given an ample class and , for any cube of , there exists a cube of such that and .
Proof.
Consider the cube of such that and . By Theorem 3.1(6), is ample. Since is shattered by , there exists a cube in such that . Since , we are done. ∎
A concept class of VC-dimension is maximum if , i.e., if (where ). The Sandwich Lemma and Theorem 3.1(5) imply that maximum classes are ample. Analogously to ample classes, maximum classes are hereditary by taking restrictions, reductions, twistings, and complements. Basic examples of maximum classes are concept classes derived from arrangements of hyperplanes in general position, balls in , and unions of intervals on the line [18, 15, 16, 21]. The following theorem summarizes some characterizations of maximum classes:
Theorem 3.5** ([18, 15, 16, 45]).**
The following conditions are equivalent for a concept class :
- (1)
* is maximum;* 2. (2)
* is maximum for all ;* 3. (3)
* and are maximum for all * 4. (4)
* is maximum.*
Following Kuzmin and Warmuth [23], we define a representation map for an ample class as a bijection satisfying the non-clashing condition: , for all . It was shown in [23] that the existence of a representation map for a maximum class implies an unlabeled sample compression scheme of size for . In Section 6, we show that this also holds for ample classes. Moreover we show that for ample classes, they are equivalent to -representation maps defined as follows. A -representation map for an ample class is a bijection satisfying the -non-clashing condition: , for all .
3.2. Examples
We continue with the main examples of ample classes.
3.2.1. Simplicial Complexes
The set of characteristic functions of simplices of a simplicial complex can be viewed as a concept class : is a bouquet of cubes with a common origin , one cube for each simplex of . Therefore, and since , is an ample class having as its simplicial complex [5].
3.2.2. Realizable Ample Classes.
Let be a convex set. Let , where is the sign pattern of . Lawrence [24] showed that is ample, and called ample classes representable in this manner realizable. Lawrence presented a non-realizable ample class of arising from a non-stretchable arrangement of pseudolines. It is shown in [6] that any ample class becomes realizable if instead of a convex set one considers a Menger -convex set of .
3.2.3. Median Classes.
A class is called median if for every three concepts of their median also belongs to . Median classes are ample by [5, Proposition 2]. Median classes are closed by taking reductions, restrictions, intersections with cubes, and products but not under complementation.
Due to their relationships with other discrete structures, median classes are one of the most important examples of ample classes. Median classes are equivalent to finite median graphs (a well-studied class in metric graph theory, see [4]), to CAT(0) cube complexes, i.e., cube complexes of global nonpositive curvature (central objects in geometric group theory, see [19, 36]), and to the domains of event structures (a basic model in concurrency theory [32, 48]).
3.2.4. Convex Geometries and Conditional Antimatroids.
Let be a concept class such that (i) and (ii) implies that . A point is called extremal if . The set of extremal points of is denoted by . A concept is generated by if is the smallest concept of containing . A concept class satisfying (i) and (ii) with the additional property that every concept of is generated by its extremal points is called a conditional antimatroid [5, Section 3]. If , then we obtain the well-known structure of a convex geometry (called also an antimatroid) [14] (See Figure 2 for an example). It was shown in [5, Proposition 1] that if is a conditional antimatroid, then , since coincides with the sets of extremal points and coincides with the set of all minimal generating sets of sets from . Hence, any conditional antimatroid is ample. Besides convex geometries, median classes are also conditional antimatroids. Another example of conditional antimatroids is given by the set of all strict partial orders on a set . Each partial order is an asymmetric, transitive subset of . Then it is shown in [5] that for any , is the set of covering pairs of (i.e., the pairs such that and there is no with ) and that
[TABLE]
Convex geometries comprise many interesting and important examples from geometry, ordered sets, and graphs, see the foundational paper [14]. For example, by the Krein-Milman theorem, any polytope of is the convex hull of its extremal points. A realizable convex geometry is a convex geometry such that its point set can be realized as a finite set of and if and only if is the intersection of a convex set of with . Acyclic oriented geometries (acyclic oriented matroids with no two point circuits) are examples of convex geometries, generalizing the realizable ones.
We continue with two particular examples of conditional antimatroids.
Example 3.6**.**
Closer to usual examples from machine learning, let be a finite set of points in , no two points sharing the same coordinate, and let the concept class consist of all intersections of axis-parallel boxes of with . Then is a convex geometry: for each , consists of all points of minimizing or maximizing one of the coordinates. Clearly, for any , there exists a box such that .
Example 3.7**.**
A partial linear space is a pair consisting of a finite set whose elements are called points and a family of subsets of , whose elements are called lines, such that any line contains at least two points and any two points belong to at most one line. The projective plane (any pair of points belong to a common line and any two lines intersect in exactly one point) is a standard example, but partial linear spaces comprise many more examples. The concept class has VC-dimension at most 2 because any two points belong to at most one line. Now, for each line fix an arbitrary total order of its points. Let consist of all subsets of points that belong to a common line and define an interval of . Then is still a concept class of VC-dimension 2. Moreover, is a conditional antimatroid: if and is an interval of the line , then consists of the two end-points of on .
3.2.5. Ample Classes from Graph Orientations.
Kozma and Moran [22] used the sandwich lemma to derive several properties of graph orientations. They also presented two examples of ample classes related to distances and flows in networks (see also [24, p.157] for another example of a similar nature). Let be an undirected simple graph and let be a fixed reference orientation of . To an arbitrary orientation of associate a concept consisting of all edges which are oriented in the same way by and by . It is proven in [22, Theorem 26] that if each edge of has a non-negative capacity, a source and a sink are fixed, then for any flow-value , the set of all orientations of for which there exists an -flow of value at least is an ample class. An analogous result was obtained if instead of the flow between and one considers the distance between those two nodes.
4. Corner Peelings and Partial Shellings
In this section, we prove that corner peelings of ample classes are equivalent to isometric orderings of as well as to partial shellings of the cross-polytope. This equivalence, combined with a result by Hall [20] yields a maximum class with VC dimension 3 without corners (Theorem 4.5 below). We show that this counterexample also refutes the analysis of the Tail Matching Algorithm of Kuzmin and Warmuth [23] for constructing unlabeled sample compression schemes for maximum classes. On the positive side, we prove the existence of corner peelings for conditional antimatroids and 2-dimensional ample classes. Finally we show that the cube complexes of all ample classes are collapsible.
4.1. Corners, Isometric Orderings, and Partial Shellings
We first establish some properties satisfied by the corners of an ample class. For , let be the smallest cube of containing and all neighbors of in that are in . Note that the dimension of is the number of neighbors of in .
Lemma 4.1**.**
Let be ample. Then:
- (i)
If then . 2. (ii)
If and is isometric then is ample and is a corner of . 3. (iii)
* is a corner of if and only if is ample.*
Proof.
Item (i): Suppose . Pick that is closest to in (with respect to the Hamming distance of ). Then and are not adjacent (by the definition of ). By the choice of , , i.e., . However, by Theorem 3.1, is ample and thus isometric by Theorem 3.2, contradicting that .
Item (ii): To prove that is ample, we use Theorem 3.2(2). First note that by item (i), . Let be parallel cubes of . If , then a gallery connecting and in is a gallery in . So, assume . If is a proper face of , then is parallel to a face of not containing . Since and are connected in by a gallery and and are connected in by a gallery, we obtain a gallery between and in . Finally, let . In this case, we assert that does not exist. Otherwise, we define a parallelism map between the concepts of and the concepts of as follows: for any , is the unique concept such that (recall that ). Note that for any : . Since is isometric, and can be connected in by a path of length . Let be the neighbor of in . Since it follows that . So, is a concept in that is closer to than ; this contradicts that .
Item (iii): If is a corner then there is a unique maximal cube containing it. Combined with Corollary 3.3, this implies that . Next, since , we get that , and by Theorem 3.1 is ample. Conversely, if is ample, applying Item (ii) to and , since is ample (and thus isometric), we get that is a corner of . ∎
Let be an ordering of the concepts in . For any , let denote the ’th level set. The ordering is called:
- •
an ample ordering if every level set is ample;
- •
a corner peeling if every is a corner of ;
- •
an isometric ordering if every level set is isometric;
- •
a locally isometric ordering if every level set is locally isometric.
Proposition 4.2**.**
The following conditions are equivalent for an ordering of an isometric class :
- (1)
* is ample;* 2. (2)
* is a corner peeling;* 3. (3)
* is isometric;* 4. (4)
* is locally isometric.*
Proof.
Clearly, (3)(4). Conversely, suppose is locally isometric but one of its levels is not isometric. Hence, there exists such that any shortest -path in contains some with . Additionally, assume that minimizes the distance among all such pairs. Since is locally isometric, necessarily . Let be the first vertex among lying in . If or (say the first), then one can replace by , which contradicts the choice of . Thus, , and at least one of them equals (say ). Now, weak isometricity implies that and have a common neighbor with . If then contradicts the minimality of , and if then contradicts the minimality of . This shows (4)(3).
If is a corner of , then any two neighbors of in have a second common neighbor in , and therefore is the restriction of on . Since is isometric, this proves (2)(3).
To show (1)(2), let be an ample ordering of . We assert that each is a corner of . Indeed, since is ample and , by Item (i) in Lemma 4.1 the cube , defined with respect to , is included in . Thus, belongs to a unique maximal cube of , i.e., is a corner of . To prove (3)(1), let be an isometric ordering. The ampleness of each follows by induction from Item (ii) of Lemma 4.1. ∎
An isometric concept class is dismantlable if it admits an ordering satisfying any of the equivalent conditions (1)-(4) in Proposition 4.2. Isometric orderings of are closely related to shellings of its dual, the cross-polytope (which we define next). Define ; so, , and we call antipodal. The -dimensional cross-polytope is the pure simplicial complex of dimension whose facets are all that contain exactly one element in each antipodal pair. Thus, has facets and each facet of corresponds to a vertex of ( if and only if ). Observe that if and only if where correspond to and corresponds to .
Let be a pure simplicial complex of dimension . Two facets are adjacent if . A shelling of is an ordering of all of its facets such that is a pure simplicial complex of dimension for every [49, Lecture 8]. A partial shelling is an ordering of some facets that satisfies the above condition. Note that is a partial shelling if and only if for every there exists such that , and , i.e., is a facet of both and . A partial shelling of is 1-step extendable if there exists a facet of such that is a partial shelling of . A partial shelling of is extendable if it can be extended to a shelling of . A pure simplicial complex is extendably shellable if every partial shelling is extendable. We next establish a relationship between partial shellings and isometric orderings.
Proposition 4.3**.**
Every partial shelling of the cross-polytope defines an isometric ordering of the corresponding vertices of the cube . Conversely, if is an isometric class of , then any isometric ordering of defines a partial shelling of .
Proof.
Let be a partial shelling of and be the ordering of the corresponding vertices of . We need to prove that each level set is isometric. It suffices to show that for every there is such that and . Equivalently, for every , there is such that and : since is a partial shelling, there is a facet with such that and . We claim that is the desired facet. It remains to show that (i) and (ii) . Item (i) follows since , and . For Item (ii), let and . We need to show that , or equivalently that . The latter follows since and .
Conversely, let be an isometric ordering and be the ordering of the corresponding facets of . We assert that this is a partial shelling. Let . It suffices to exhibit such that and . Since is isometric, has a neighbor . Since it follows that . Since it follows that and thus . ∎
Corollary 4.4**.**
For any partial shelling , let be the ordering of the corresponding vertices of and let . Then the following hold.
- (1)
* and are ample;* 2. (2)
the partial shelling is 1-step extendable if and only if has a corner. 3. (3)
the partial shelling is extendable if and only if is dismantlable.
Consequently, if all ample classes are dismantlable, then is extendably shellable.
Proof.
By Proposition 4.3, the level sets , are isometric, thus is ample by Proposition 4.2. The complement is also ample, establishing (1).
If has a corner, then by Lemma 4.1(iii), contains a concept such that is ample. Consequently, its complement is ample. Let be the facet of corresponding to . By Proposition 4.2 is an isometric ordering of . by Proposition 4.3, is a partial shelling of and thus is 1-step extendable. Conversely, assume that there exists such that is a partial shelling of and let be the vertex of corresponding to . By Proposition 4.3, is an isometric ordering of . By Proposition 4.2, is ample and thus its complement is also ample, establishing that is a corner of by Lemma 4.1(iii). This establishes (2).
Assertion (3) is a direct consequence of Assertion (2). Consequently, if all ample classes are dismantlable, then any partial shelling of is extendable and thus is extendably shellable. ∎
It was asked in [49] if any cross-polytope is extendably shellable. In the PhD thesis of H. Tracy Hall from 2004, a nice counterexample to this question is given [20]. Hall’s counterexample arises from the 299 regions of an arrangement of 12 pseudo-hyperplanes. These regions are encoded as facets of the 12-dimensional cross-polytope and it is shown in [20] that the subcomplex of consisting of all other facets admits a shelling which cannot be extended by adding any of the 299 simplices. By Corollary 4.4(ii), the ample concept class defined by those 299 simplices does not have any corner (see Fig. 3 for a picture of ).555For the interested reader, a file containing the 299 concepts of represented as elements of is available at https://arxiv.org/src/1812.02099/anc/CH.txt A counting shows that is a maximum class of VC-dimension 3. This completes the proof of our first main result:
Theorem 4.5**.**
There exists a maximum class of VC-dimension 3 without any corner.
Remark 4.6**.**
Hall’s concept class also provides a counterexample to [30, Conjecture 4.2] asserting that for any two ample classes with there exists an ample class such that . As noticed in [30], this conjecture is stronger than the corner peeling conjecture disproved by Theorem 4.5.
Remark 4.7**.**
Notice also that since ample classes are closed by Cartesian products, and any corner in a Cartesian product comes from corners in each factor, one can construct other examples of ample classes without corners by taking the Cartesian product of by any ample class.
4.2. refutes the Analysis by Kuzmin and
Warmuth [23]
The algorithm of [23] uses the notion of forbidden labels for maximum classes, introduced in the PhD thesis of Floyd [15] and used in [16, 23]; we closely follow [15]. Let be a maximum class of VC-dimension on the set . For any with , the restriction is a maximum class of dimension . Thus contains concepts. There are possible concepts on . We call the characteristic function of the unique concept that is not a member of a forbidden label of size for . Each forbidden label forbids all concepts that contain it from belonging to . Let be a concept which contains the forbidden label for . Since is a maximum class, adding to would shatter the set that is of cardinality .
The algorithm of [23], called the Tail Matching Algorithm, recursively constructs a representation map for , expands to a map on the carrier of , and extends to using a special subroutine. This subroutine and the correctness proof of the algorithm (Theorem 11) heavily uses that a specially defined bipartite graph (which we will call ) has a unique perfect matching. This bipartite graph has the concepts of on one side and the forbidden labels of size for on another side. Both sides have the same size. There is an edge between a concept and a forbidden label if and only if the forbidden label is contained in the respective concept. The graph is defined in [23, Theorem 10], which also asserts that has a unique perfect matching. In the following, we show that this assertion is false.
We will show that uniqueness of the matching implies that there is a corner in the tail (which is contradicted by Hall’s concept class ). We use the following lemma about perfect bipartite matchings:
Lemma 4.8** ([10]).**
Let be a bipartite graph with bipartition and unique perfect matching . Then, there are vertices and with degree one.
By Lemma 4.8, the uniqueness of the matching claimed in [23, Theorem 10] implies that there is a forbidden labeling that is contained in exactly one concept of the tail. We claim that must be a corner: is the only concept in the tail realizing this forbidden labeling and removing this concept from reduces the number of shattered sets by at least one. After the removal, the number of concepts is . By the Sandwich Lemma, the number of shattered sets is always at least as big as the number of concepts. So removing reduces the number of shattered sets by exactly one set and the resulting class is ample. For ample classes, the number of concepts equals the number of supports of cubes of the 1-inclusion graph and for every shattered set there is a cube with this shattered set as its support. Thus lies in a cube with support . There is no other cube with support because after removing there is no cube left with support . Thus is the unique cube with support and must be maximal. We conclude that is a corner of . Since is maximum and does not contain corners, this leads to a contradiction. This establishes that Theorem 10 from [23] is false.
4.3. Two Families of Dismantlable Ample
Classes
We continue with two families of dismantlable ample classes: conditional antimatroids and 2-dimensional ample classes.
Proposition 4.9**.**
Conditional antimatroids are dismantlable.
Proof.
Let be an ordering of the concepts of , where if and only if (breaking ties arbitrarily). Clearly, and the order is monotone with respect to distances from . In particular, any order defined by a Breadth First Search (BFS) from satisfies this condition. We prove that for each , is a corner of the level set . The neighbors of in are subsets of containing elements. From the definition of extremal points of that we denote by , it immediately follows that if and only if . For any , since and is closed under intersections, we conclude that . Therefore, the whole Boolean cube between and is included in , showing that is a corner of . ∎
The fact that 2-dimensional maximum classes have corners was proved in [35, Theorem 34] and it was generalized in [28] to 2-dimensional ample classes. We provide here a different proof of this result, originating from 1997-1998 and based on a local characterization of convex sets of general ample classes, which may be of independent interest.
Given two classes , is convex in if for any and is locally convex in if for any with .
Lemma 4.10**.**
A connected subclass of an ample class is convex in if and only if is locally convex.
Proof.
One direction of the statement is trivial. For the other direction, assume that is locally convex. For , recall that denotes the distance between and in . Recall also that since is ample, is isometric and thus for any . We prove that for any , by induction on , the case being covered by the initial assumption. Pick any and let be a shortest -path of passing via . By Theorem 3.2, is contained in . Let be a shortest -path in . Let be the neighbor of in . Since , by the induction assumption . Since is bipartite, either or . In the first case, and we are done. So suppose that . This implies that is a shortest -path in . Moreover, we can assume that and consequently, and the edge is parallel to an edge of the path . Consider a shortest gallery connecting the edges and in . It is constituted of two shortest paths and . Then together with the subpath of comprised between and constitute a shortest path between and , thus it belongs to . Therefore, if is comprised in between and , then we are done. Thus suppose that belongs to the subpath of between and . Since is adjacent to , by local convexity of we obtain that . Applying this argument several times, we deduce that the whole path belongs to . In particular, . Since is between and , , thus . By induction hypothesis, . Since , we are done. ∎
Proposition 4.11**.**
2-dimensional ample classes are dismantlable.
Proof.
As for conditional antimatroids, the corner peeling for -dimensional classes is based on an algorithmic order of concepts. Consider the following total order of the concepts of a -dimensional ample class constructed recursively as follows: start with an arbitrary concept and denote it and at step , having numbered the concepts , select as a concept in which is adjacent to a maximum number of concepts of . We assert that is ample and that is a corner of . Since is ample and is 2-dimensional, by Lemma 4.1, has at most two neighbors in . First suppose that has two neighbors and in . By Lemma 4.1, and are included in a 2-cube with a fourth concept belonging to . Since is a maximal cube of , by Corollary 3.3, is not parallel to any other cube of . On the other hand, the edges and are parallel to the edges and , respectively. This shows that is connected for any , and therefore by Theorem 3.2(2), is ample and is a corner. Now suppose that has exactly one neighbor in . From the choice of at step , any concept of has at most one neighbor in , i.e., is locally convex. By Lemma 4.10, is convex in . But then is isometric since has degree in . By Lemma 4.1, is thus ample and is a corner of . ∎
4.4. Collapsibility
A free face of a cube complex is a face of strictly contained in only one other face of . An elementary collapse is the deletion of a free face (thus also of ) from . A cube complex is collapsible to a vertex if can be reduced to by a sequence of elementary collapses. Namely, there exists an ordered sequence where each is a free face of contained in and is . Observe that each face of distinct from appears exactly once in the sequence.
Collapsibility is a stronger version of contractibility. The sequences of elementary collapses of a collapsible cube complex can be viewed as discrete Morse functions [17] without critical cells, i.e., acyclic perfect matchings of the face poset of . From the definition it follows that if has a corner peeling, then the cube complex is collapsible: the sequence of elementary collapses follows the corner peeling order (in general, detecting if a finite complex is collapsible is NP-complete [42]). Theorem 3.2(5) implies that the cube complexes of ample classes are contractible (see also [7] for a more general result). In fact, the cube complexes of ample classes are collapsible (this extends the collapsibility of finite CAT(0) cube complexes established in [1]):
Proposition 4.12**.**
If is an ample class, then the cube complex is collapsible.
Proof.
We proceed by induction on the size of . If contains only one concept, then the statement is trivially true. Otherwise, let and suppose by induction hypothesis that is collapsible to . Let be the corresponding collapsing sequence of , i.e., a partition of the faces of into pairs such that is a free face in the current subcomplex of and is the unique face properly containing .
Each cube of is mapped to a cube of with . Observe that if and otherwise. In the first case, is contained in and is entirely contained in the carrier . Conversely, by Lemma 3.4, any cube of is the image of at least one cube of (with ). If there exists such that , then there exist exactly three cubes of that are mapped to : and the two opposite facets and of such that . Otherwise, there exists a unique cube of that is mapped to .
We derive a collapsing sequence for by replacing each elementary collapse of by one, two, or three elementary collapses in and when , we add a last elementary collapse. Consider a pair . If neither nor are contained in , let and be the unique preimages of and in and insert the pair in .
If both and are contained in , let be the cubes of mapped to and the ones mapped to such that , , is contained in , is contained in , and is contained in . We insert the three pairs in in this order.
Suppose now that is included in and is not included in . Let be the unique cube of mapped to and let be the cubes of mapped to such that . Assume without loss of generality that is a facet of both and . We insert the two pairs in in this order.
Finally, we consider the vertex . If , then the preimage of contains only one vertex that we denote by . Suppose now that . Let such that and . Then the preimage of in is of size : it contains the vertices and the edge . In this case, we add to the elementary collapse .
Each cube of is in the preimage of a single cube of . Moreover, each cube of appears in exactly one collapsing pair of the sequence . Consequently, each cube of whose image is not appears exactly once in the sequence . Notice also that when the preimage of is of size , both and appear in the last pair of . Consequently, each cube of appears in exactly one pair of the sequence . Since each pair of (except potentially ) is derived from a collapsing pair of and since is a sequence of elementary collapses of , we can deduce that is a sequence of elementary collapses of the cube complex to . ∎
Example 4.13**.**
We illustrate the construction in the proof of Proposition 4.12 with the ample class from Figure 4 and its restriction . The names of the concepts are depicted in the figure. The faces will be denoted by the list of their vertices.
Consider the following two collapsing sequences of :
- •
, and
- •
.
Note that collapses to and collapses to .
The corresponding collapsing sequences for obtained by the construction described in the proof of Proposition 4.12 are the following:
- •
collapses to , and
- •
collapses to .
In both cases, the subsequence corresponds to the elementary collapse . In , the subsequence corresponds to the elementary collapse in . In , the elementary collapse corresponds to the elementary collapse in and the elementary collapse is the elementary collapse associated to the last vertex of .
5. Representation Maps for Maximum Classes
In this section, we prove that maximum classes admit representation maps, and therefore optimal unlabeled sample compression schemes.
Theorem 5.1**.**
Any maximum class of VC-dimension admits a representation map, and consequently, an unlabeled sample compression scheme of size .
The crux of the proof of Theorem 5.1 is the following proposition. Let be a -dimensional maximum class and let be a -dimensional maximum subclass. A missed simplex for the pair is a simplex . Note that since and are maximum, any missed simplex has size . An incomplete cube for is a cube of such that is a missed simplex. For any incomplete cube with , is a -cube and is a maximum class of dimension . Observe that any incomplete cube for is a maximal cube in and by Corollary 3.3, there is a bijection between the missed simplices for and the incomplete cubes for .
Consider a missed simplex for and the incomplete cube for such that . Since , we have . Since , there exists a unique concept such that . We call the source of and we consider the source-map from the set of incomplete cubes for to where is the source of . In fact, we show in the following proposition that the source-map is a bijection between the incomplete cubes for and the concepts of .
Proposition 5.2**.**
Each is the source of a unique incomplete cube for . Moreover, if is a representation map for and extends by setting for each , then is a representation map for .
Proof of Theorem 5.1..
Following the general recursive construction idea of [23], we derive a representation map for by induction on . If , then is a [math]-dimensional maximum class containing a unique concept . In this case, we set and is clearly a representation map for . For the induction step, (see Figure 5), pick and consider the maximum classes and with domain . By induction, has a representation map . Use Proposition 5.2 to extend to a representation map of . Define a map on as follows:
[TABLE]
It is easy to verify that is non-clashing: indeed, if satisfy then . Since , it follows that also disagree on . Else, and . In this case, and therefore disagree on .
It remains to show that is a bijection between and . It is easy to verify that is injective. So, it remains to show that , for every . This is clear when or . If and , then and (since is -dimensional). Hence, as required, concluding the proof. ∎
Proof of Proposition 5.2..
To prove the proposition, we first prove that incomplete cubes and their sources are preserved by restrictions and reductions (Lemma 5.4). To show that each concept of is the source of one incomplete cube, we consider a minimal counterexample and we establish that in this counterexample each concept is the source of at most incomplete cubes. Moreover, if a concept is the source of [math] (respectively, ) incomplete cubes, then any of its neighbors in is the source of (respectively, ) incomplete cubes (Lemma 5.5). Using this and the notions of galleries and the associated trees defined below, we establish the first assertion of the proposition. Then using Lemma 5.4 and the first assertion, we establish the second part of the proposition. We also give a geometric characterization of sources (Lemma 5.3) that is used later to estimate the complexity of computing the representation map (see Remark 5.7).
Call a maximal cube of a chamber and a facet of a chamber a panel (a -panel if its support is ). Any -panel in satisfies and . Recall that a gallery between two parallel cubes (say, two -cubes) is any simple path of -cubes , where is a -cube. By Theorem 3.2(3), any two parallel cubes of are connected by a gallery in . Since is a maximum class, any panel of is parallel to a panel that is a maximal cube of . Also for any maximal simplex , the class is a maximum class of dimension 1 and is a maximum class of dimension 0 (single concept). Thus is a tree (e.g. [18, Lemma 7]) which contains the unique concept . We call the root of and we denote by the unique -panel of such that .
The next result provides a geometric characterization of sources:
Lemma 5.3**.**
Let be a missed simplex of the pair . A concept is the source of the unique -cube if and only if for any , if and are the two -panels of with , then is on the path between and the root of the tree .
Proof.
Observe that there exists a unique concept such that for any , if and are the two -panels of with , then is on the path between and the root of the tree . Since contains a unique source, it is enough to show that this unique concept is the source.
Assume by way of contradiction that this is not the case, i.e., that is the source of and that there exists and two -panels with and such that the unique gallery between and the root passes via , i.e., . Since is a maximal cube, by Corollary 3.3, is not in the domain of the chamber for . This implies that there exists such that , and consequently, is not the missed sample for . ∎
In the next lemma, we show that incomplete cubes and their sources are preserved by restrictions and reductions.
Lemma 5.4**.**
Let be an incomplete cube for with source and support , and let such that and . Then, the following holds:
- (i)
* is an incomplete cube for whose source is .* 2. (ii)
* is an incomplete cube for whose source is .*
Proof.
Item (i): and are maximum classes on of VC-dimensions and , and . Therefore, is an incomplete cube for . By definition, is the unique concept such that . Since , and is the unique concept of so that , i.e., is the source of .
Item (ii): and are maximum classes on of VC-dimensions and . Since , and is an incomplete cube for . Let . It remains to show that . Indeed, otherwise both extensions of in , namely , are in which contradicts that . ∎
Next we prove that each concept of is the source of a unique incomplete cube. Since there is a bijection between incomplete cubes and missed simplices, the number of incomplete cubes is . Therefore, it is sufficient to show that each concept of is the source of at most one incomplete cube. Assume the contrary and let be a counterexample minimizing the size of . First, if a concept is the source of two incomplete cubes , then . Indeed, let and . By Lemma 5.4(i) and minimality of , . Indeed, if there exists , is the source of the incomplete cubes and for , contrary to minimality of . By Lemma 5.4(ii) and minimality of , . Indeed, if there exists , is the source of the incomplete cubes and for , contrary to minimality of .
Next we assert that any is the source of at most 2 incomplete cubes. Indeed, let be the source of incomplete cubes . Then , i.e., . For similar reasons, . Thus, by Corollary 3.3, .
Lemma 5.5**.**
Let be neighbors and let . Then, is the source of 2 incomplete cubes if and only if is the source of 0 incomplete cubes. Consequently, every connected component in either contains only concepts with , or only concepts with .
Proof.
By minimality of , is the source of a unique incomplete cube for and is the source of a unique incomplete cube for . Let be the incomplete cube for such that is the source of . Let be the incomplete cube for such that is the source of . Since has a unique source , by Lemma 5.4(i), and consequently, . Similarly, since has a unique source , by Lemma 5.4(ii), and thus . Consequently, is the source of 2 incomplete cubes ( and ) if and only if is the source of 0 incomplete cubes. ∎
Pick that is the source of two incomplete cubes for and an incomplete cube such that . Let , , and . The concept belongs to a unique -panel . Let be the unique gallery between the root of the tree and . For , denote the chamber by . Since is ample and is not contained in , it follows that the complement is a nonempty ample class. Hence induces a nonempty connected subgraph of . Therefore, it follows that and each concept are connected by a path in , and by Lemma 5.5 it follows that
[TABLE]
Consider the chamber and its source . By the definition of the source, necessarily and . Therefore, Property (5.1) implies that there must exist another cube such that . Let be the neighbor of in ; note that . Since , it follows that , contradicting that . This establishes the first assertion of Proposition 5.2.
We prove now that the map defined in Proposition 5.2 is a representation map for . It is easy to verify that it is a bijection between and , so it remain to establish the non-clashing property: for all distinct pairs . This holds when because is a representation map. Next, if and , this holds because by the properties of .
Thus, it remains to show that every distinct satisfy the non-clashing condition. Assume towards contradiction that this does not hold and consider a counterexample with minimal domain size . Consequently, there exist distinct such that for any , where and . Since , there exists such that . If there exists distinct from , then from Lemma 5.4(1), and are incomplete cubes for whose respective sources are and . Since , , and (since they differ on ), and clash in , contradicting the minimality of the counterexample . Consequently, we can assume that and that and differ only on . In this case, by Lemma 5.4(1), is the source of both incomplete cubes and . Since and are distinct incomplete cubes, we have and thus, is the source of two different incomplete cubes of , contradicting the first statement of the proposition. This ends the proof of Proposition 5.2. ∎
Remark 5.6**.**
Proposition 5.2 relies on a canonical bijection between missed simplices and incomplete cubes (that allows to define the source-map). This is due to the fact that any missed simplex for is a maximal simplex of and is thus the support of a unique cube in . This property does not longer hold for general ample classes since there are missed simplices that are not maximal and therefore there are the supports of several cubes in . A first step to generalize Proposition 5.2 to ample classes could be to find a way to define a source-map between the missed simplices in and the concepts in .
Remark 5.7**.**
To compute the representation map for a maximum class of dimension , we make recursive calls. For each call, the costliest operation is to compute the source-map that can be done in time as follows. First we naively compute in the cubes of dimension in and the cubes of dimension in . Then we compute in time the trees for any maximal simplex (the roots of those trees are the cubes of of dimension ) and we can then compute the source of each cube by Lemma 5.3. Consequently, one can compute a representation map for a maximum class of dimension in .
Remark 5.8**.**
In the appendix, we give two representation maps for Hall’s concept class presented in Figure 3. One of this representation map is obtained by the method described in the proof of Theorem 5.1. The other one is obtained by transforming the problem into a SAT formula and using a SAT solver.
6. Representation Maps for Ample Classes
In this section, we provide combinatorial and geometric characterizations of representation maps of ample classes. We first show that representation maps lead to optimal unlabeled sample compression schemes and that they are equivalent to unique sink orientations (USO) of (see below for the two conditions defining USOs). We also show that corner peelings of ample classes are equivalent to the existence of acyclic USOs. In Section 6.3, we show how from representation maps for an ample class to derive representation maps for substructures of (intersections with cubes, restrictions and reductions). In Section 6.4, we show that there exist maps satisfying each one of the two conditions defining USOs (but not both). Finally, using the geometric characterization of representation maps as USOs, we show that constructing a representation map for an ample concept class can be reduced to solving an instance of the Independent System of Representatives problem [2].
6.1. Unlabeled Sample Compression Schemes and Representation Maps
In the next theorem, we prove that, analogously to maximum classes, representation maps for ample classes lead to unlabeled sample compression schemes of size . This also shows that the representation maps for ample classes are equivalent to -representation maps.
Theorem 6.1**.**
Let be an ample class and let be a bijection. The following conditions are equivalent:
- (R1)
-non-clashing**: For all distinct concepts , . 2. (R2)
Reconstruction*: For every realizable sample of , there is a unique that is consistent with and .* 3. (R3)
Cube injective*: For every cube of , the map is an injection from to .* 4. (R4)
-non-clashing**: For all distinct concepts , .
Moreover, any -non-clashing map is bijective and is therefore a representation map. Furthermore, if is a representation map for , then there exists an unlabeled sample compression scheme for of size .
Proof.
Fix and partition into equivalence classes where two concepts are equivalent if . Thus, each equivalence class corresponds to a sample of with domain , i.e., a concept in . We first show that the number of such equivalence classes equals the number of concepts whose representation set is contained in :
[TABLE]
Condition (R2) asserts that in each equivalence class there is exactly one concept such that .
(R1) (R2): Assume and consider a sample for which the property does not hold. This implies that there exists an equivalence class with either zero or (at least) two equivalent concepts for which with . Note that since the number of equivalence classes equals the number of concepts whose representation set is contained in , if some equivalence class has no concept for which , then there must be another equivalence class with two distinct concepts for which . Therefore, in both cases, there exist two equivalent concepts such that . Since , we have , contradicting .
(R2) (R1): Assume , i.e., for two distinct concepts , we have . Now for the sample , we have . Furthermore, and . This implies .
(R1)(R2) (R3): Since is ample, it suffices to show that for every there is some with . This is established by the following claim.
Claim 6.2**.**
Conditions (R1) and (R2) together imply that for any , there exists a unique concept such that .
Proof.
For any concept , let . Let . Note that all concepts in agree on domain : indeed, is the single sample . We prove the claim by induction on .
Suppose first that . Since , and is a sample of . By condition (R2), there is a unique concept such that (i) (i.e., ), and (ii) (i.e., ). Thus choosing settles this case.
Assume now that and that for every , there is a unique with .
We assert that for any two distinct concepts such that , we have . Indeed, suppose by way of contradiction that and note that . If , then
[TABLE]
Since , this implies that , contradicting condition (R1).
Consequently, all the samples are pairwise distinct. There are such samples and each sample corresponds to a proper subset of . By the previous assertion, there exist at most concepts such that , and thus there exists at most one concept such that .
Thus, it remains to establish the existence of such that . Since there are ’s for , it follows that there is a unique sample with that is not realized by any of the ’s. Consider the sample with domain defined by
[TABLE]
Since is shattered by , it follows that is realized by . By condition (R2) there is a unique concept that agrees with such that (i.e. ). We claim that is the desired concept . First notice that , because agrees with on . Since is not realized by any , , and since is the unique concept of such that (by induction hypothesis), necessarily we have that for any . Consequently, , concluding the proof of the claim. ∎
(R3) (R4): For any distinct concepts , consider the minimal cube which contains both . This means that for every , and that for every . Condition (R3) guarantees that the map is an injection from to . Therefore . It follows that there must be some such that . Since , and therefore and condition (R4) holds for and .
(R4) (R1): This is immediate because if two concepts clash on their symmetric difference, then they also clash on their union.
Moreover, observe that for any map , if for , then and is not -non-clashing. Consequently, any -non-clashing map is injective and thus bijective since .
We now show that if is a representation map for then there exists an unlabeled sample compression scheme for . Indeed, by (R2), for each realizable sample , let be the unique concept such that and . Then consider the compressor such that for any , and the reconstructor such that for any , . Observe that by the definition of , and coincides with on . Consequently, and defines an unlabeled sample compression scheme for of size . This concludes the proof of the theorem. ∎
By applying (R3) to the -dimensional cubes of , we get the following corollary.
Corollary 6.3**.**
For any representation map of an ample class , for any and , we have .
6.2. Representation Maps as Unique Sink Orientations
We call a map edge-non-clashing if for any -edge , . Note that defines an orientation of the edges of : an -edge is oriented from to if and only if . Conversely, given an orientation of the edges of , the out-map of associates to each the coordinate set of the edges outgoing from . Note that the out-map of any orientation is edge-non-clashing. Note also that any -representation map is edge-non-clashing and thus defines an orientation of . Moreover, by Corollary 6.3, the out-map of coincides with .
An orientation of the edges of (or the corresponding out-map ) is a unique sink orientation (USO) if it satisfies the following two conditions.
- (C1)
for any , the cube of which has support and contains is a cube of , i.e., all outgoing neighbors of belong to a cube of ; 2. (C2)
For any cube of , there exists a unique such that , i.e., is a sink in .
Observe also that a map satisfying Condition (C2) is necessarily an edge-non-clashing map. If is a cube, then Condition (C1) trivially holds and Condition (C2) corresponds to the usual definition of USOs on cubes [41]. In the following, we use the characterization of USOs for cubes given in [41].
Lemma 6.4** ([41]).**
For a cube and a map , the following are equivalent:
- (1)
* is the out-map of a unique sink orientation of ;* 2. (2)
* is -non-clashing;* 3. (3)
for any subcube of , the map is a bijection between and ; 4. (4)
* is the out-map of a unique source orientation of , i.e., for any subcube of , there exists a unique such that .*
The equivalences (1)(2) and (1)(4) are respectively [41, Lemma 2.3] and [41, Lemma 2.1]. The implications (2)(3) and (3)(1) are trivial.
Remark 6.5**.**
In view of Lemma 6.4, Condition (C2) looks similar to Condition (R3) of Theorem 6.1. Note however that (C2) is about the cubes of contained in while (R3) is about all cubes of . Similarly, Condition (C2) implies that is -non-clashing on each cube of contained in while condition (R4) of Theorem 6.1 requires that is -non-clashing on .
Remark 6.6**.**
If is an ample class and is a USO of , then for any cube of , the restriction of to the edges of trivially satisfies (C1) and (C2) and is thus a USO of . Consequently, its out-map from to satisfies the conditions of Lemma 6.4.
We now show that representation maps for ample classes give rise to USOs.
Corollary 6.7**.**
*If is a representation map for an ample class , then is a unique sink orientation. *
Proof.
Pick some concept and consider the unique cube of that contains and has support . Since is a representation map, by Condition (R3) of Theorem 6.1, the map is a bijection between the ample set and . Consequently, , which is possible only if . This proves (C1). Condition (C2) follows from Condition (R4) of Theorem 6.1 applied to the cubes of and Lemma 6.4. ∎
We continue with a characterization of representation maps of ample classes as out-maps of USOs, extending a similar result of Szabó and Welzl [41] for cubes. This characterization is “local-to-global”, since (C1) and (C2) are conditions on the cubes around each concept .
Theorem 6.8**.**
For an ample class and a map , the following are equivalent:
- (1)
* is a representation map;* 2. (2)
* is the out-map of a USO;* 3. (3)
* for any and satisfies (C2).*
Before proving the theorem, starting from an edge-non-clashing map for an ample class , we show how to derive maps for restrictions , reductions and intersections with cubes of . Consider an edge-non-clashing map for an ample class . Given a cube of , define by setting for any . Note that is the out-map of the orientation restricted to the edges of .
Given , define by
[TABLE]
Hence, for each with . Consequently, for an -edge of between and , is the label of the origin of this edge minus ; we call the -out-map of .
Given , define by
[TABLE]
Hence, for each with . Consequently, for an -edge of between and , is the label of the destination of this edge; we call the -in-map of .
In the next lemma, we show that if the map is the out-map of a USO, then and are also outmaps of USOs. A similar result holds for but it will be proved in Section 6.3. The first assertion of the lemma generalizes Remark 6.6.
Lemma 6.9**.**
Consider a map that is the out-map of a USO of an ample class . For any cube and any , the following hold:
- (1)
* is the out-map of a USO of ;* 2. (2)
* is the out-map of a USO of ;*
Proof.
Item (1): Consider a cube . Since (C1) holds for and , for any concept , the cube that has support and contains is a cube of . Since the intersection of two cubes is a cube, the cube has support and contains . Consequently, (C1) holds for and . Since any cube of is also a cube of and since (C2) holds for and , (C2) also holds for and .
Item (2): Consider and let such that . By (C1), there exists an -cube containing in . Thus, there exists an -cube containing in . Consequently, satisfies (C1).
Suppose that there exists a cube in violating (C2), i.e., there exist such that . By definition of , there exist such that and . Since is a cube of containing and , there exists a cube of containing and such that . Consequently, and similarly, . Therefore, and is not injective on the cube of , contradicting Lemma 6.4 (see Remark 6.6).
∎
Proof of Theorem 6.8.
The implication (1)(2) is established in Corollary 6.7. Now, we prove (2)(1). Clearly, property (C1) implies that for any , whence is a map from to .
Let be an ample class of smallest size admitting a non-representation map satisfying (C1) and (C2). Hence there exist such that , i.e., ; is called a clashing pair.
Claim 6.10**.**
If is a clashing pair, then and .
Proof.
Since is ample and , is a clashing pair for and the restriction of to . By Lemma 6.9, is the out-map of a USO of . Consequently, by minimality of , and thus . Moreover, if , then there is and , contradicting that is a clashing pair.
Suppose and pick . Consider the -out-map . By Lemma 6.9, is the out-map of a USO for . Suppose without loss of generality that . Let and . Then , and consequently is a clashing pair for on . Since is ample, smaller than , and since is the out-map of a USO of , this contradicts the minimality of . ∎
Claim 6.11**.**
* is a cube minus a vertex.*
Proof.
By (C2) and Claim 6.10, is not a cube. If is not a cube minus a vertex, since the complement is also ample (thus is connected), contains an -edge with and . Consider the -in-map .
For any , let such that . Since satisfies (C1), belongs to an -cube in and consequently, belongs to an -cube in . Thus and satisfy (C1).
Suppose that and violate (C2). Then there exists a cube of and such that . Without loss of generality, we can assume that . Let such that and let such that . The restriction of the ample class is the cube . By Lemma 6.9, is the out-map of an USO of . Since and is an -edge, . Thus there exists such that and the edge of belong to different -half-spaces and of the cube . Since , the half-space containing also contains a concept of . Hence, is a proper ample subset of . Since , , , , we deduce that and . Since , is a clashing pair for the restriction of on , contrary to the minimality of . Consequently, and satisfy (C2) and is the outmap of an USO of . By minimality of , is a representation map for .
Consider a clashing pair for and , and let and . By Claim 6.10, . Since is a representation map for , necessarily . Consequently, , i.e., is an -edge of . This is impossible since satisfies (C2). Therefore, is necessarily a cube minus a vertex. ∎
Now, we complete the proof of the implication (2)(1). By Claim 6.10, . By condition (C1) and Claim 6.11, for any . Thus there exists a set such that for any . Every -cube of contains a source for (i.e., ). For each -cube of , let . Notice that since . Consequently, the number of distinct sets (when runs over the -cubes of ) is at most . On the other hand, since is a cube minus one vertex by Claim 6.11, there are -cubes in . Consequently, there exist two -cubes such that . Thus and is a clashing pair for and , contradicting Claim 6.10.
The implication (2)(3) is trivial. To prove (3)(2), we show by induction on that a map satisfying (C2) also satisfies (C1). For any , consider the -out-map . Recall that if is an -edge directed from to , then and maps to . Thus maps to . Moreover, each cube of is contained in a unique cube of such that . If there exist such that , then there exist such that , contradicting (C2). Consequently, satisfies (C2). By induction hypothesis, satisfies (C1) for any .
For any concept , pick . Since satisfies (C1), belongs to a -cube in with . This implies that belongs to a -cube in with . Thus satisfies (C1), concluding the proof of Theorem 6.8. ∎
A consequence of Theorems 6.1 and 6.8 is that corner peelings correspond exactly to acyclic unique sink orientations.
Proposition 6.12**.**
An ample class admits a corner peeling if and only if there exists an acyclic orientation of the edges of that is a unique sink orientation.
Proof.
Suppose that is a corner peeling and consider the orientation of where an edge is oriented from to if and only if . Clearly, this orientation is acyclic. For any , since is a corner in , the outgoing neighbors of belong to a cube of , i.e., satisfies (C1). For any cube of , assume that is the first concept of in the ordering . Observe that is a sink of for the orientation . Note that for each , . By Theorem 3.1(6), is ample and thus connected. Consequently, for each , there exists in with such that . Consequently, since is oriented from to , is not a sink of . Therefore every cube has a unique sink for the orientation and is an acyclic unique sink orientation of .
Suppose now that admits an acyclic unique sink orientation . Consider a concept that is a source for . By (C1) all the neigbors of belong to a cube of and consequently, is a corner of . Since is ample, by Lemma 4.1, is ample. Clearly the restriction of the orientation to is acyclic. We claim that is also a USO for . Observe that any cube of is also a cube of and consequently, satisfies (C2) since satisfies (C2). Suppose now that there exists such that the outgoing neighbors of in do not belong to a cube of . Then since is a USO for , necessarily the outgoing neighbors of in belong to a cube of that contains . But then and are both sources of for the orientation . By Lemma 6.4, this implies that does not satisfy (C2) on and thus on (see Remark 6.6), a contradiction. Therefore, satisfies (C1) and (C2) and is a unique sink orientation of . Applying the previous argument inductively we obtain a corner peeling of . ∎
Remark 6.13**.**
A consequence of Proposition 6.12 is that for any representation map for Hall’s concept class , the corresponding USO of contains a directed cycle and thus at least one non-trivial strongly connected component.
The USOs corresponding to the two representation maps of presented in the appendix contain one or two non-trivial strongly connected components.
6.3. Representation Maps for Substructures
In this subsection, from a representation map for an ample class , we show how to derive representation maps for restrictions , reductions and intersections with cubes of .
Given a subset , define as follow. For any , there exists a unique -cube in such that . By Lemmas 6.4 and 6.9(1), there exists a unique such that ( is the source of for ). We set . If , coincides with the -out-map defined in Section 6.2.
Given a subset , define as follow. For any , there exists a unique -cube in such that . Since , . By Claim 6.2, there exists a unique such that ( is the unique sink of for ). We set . If , coincides with the -in-map defined in Section 6.2.
Proposition 6.14**.**
For a representation map for an ample class , any cube of , and any , the following hold:
- (1)
* is a representation map for ;* 2. (2)
* is a representation map for ;* 3. (3)
* is a representation map for .*
Proof.
By Theorem 6.8, Assertion (1) follows from Assertion (1) of Lemma 6.9 and Assertion (2) follows by iteratively applying Assertion (2) of Lemma 6.9 to the elements of .
By Theorem 6.8, to prove Assertion (3), it is enough to show that is the out-map of a USO of . Recall that for any , we have , where is the unique -cube of such that and is the unique sink of for . By (C1), there exists an -cube containing in . Since , there exists an -cube of containing . Consequently, and satisfy (C1).
Suppose there exists a cube in violating (C2), i.e., there exist such that . Any cube in extends to a unique cube of such that . By definition of , there exist such that and . Consequently, since and similarly . Consequently, the map is not injective on , contradicting Condition (R3) of Theorem 6.1. ∎
6.4. Pre-representation Maps
We now show that we can find maps satisfying each of the conditions (C1) and (C2). Nevertheless, we were not able to find a map satisfying (C1) and (C2). It is surprising that, while each -cube has at least USOs [27], it is so difficult to find a single USO for ample classes.
Proposition 6.15**.**
For any ample class there exists a bijection and an injection such that satisfies the condition (C1) and satisfies the condition (C2).
Proof.
First we prove the existence of the bijection . For , denote by the union of all -cubes included in and call the carrier of . For , denote by the union of all carriers . Define a bipartite graph , where there is an edge between a concept and a strongly shattered set if and only if belongs to the carrier . We assert that admits a perfect matching . By the definition of the edges of , if the edge is in , then belongs to and thus the unique -cube containing is included in . Thus defined by setting if and only if is a bijection satisfying (C1).
Since is ample, we have , and thus to prove the existence of , we show that the graph satisfies the conditions of Philip Hall’s theorem [26]: if is an arbitrary subset of simplices of , then . Indeed, since is ample, for any the carrier contains at least one -cube, thus . Consequently, by the Sandwich Lemma applied to the class .
We now prove that there exists an injection satisfying (C2). We prove the existence of by induction on the size of . If , trivially exists. Consider now and note that there exists an injection satisfying (C2) by induction hypothesis. We define by:
[TABLE]
It means that the orientation of the edges of is obtained by keeping the orientation of the edges of and orienting all -edges of from to . It is easy to verify that is injective.
Consider two distinct concepts belonging to a common cube of and let that is the minimal cube of containing and . If , then since , since , and since satisfies (C2) on the cube of , there exists . Suppose now that . In this case, and . ∎
One can try to find representation maps for ample classes by extending the approach for maximum classes: given ample classes and with , a representation map for is called -entering if all edges with and are directed by from to . The representation map defined in Proposition 5.2 is -entering. Given , suppose that is a -entering representation map for .
We can extend the orientation to an orientation of as follows. Each -edge of is directed arbitrarily, while each other edge is directed as the edge is directed by . Since satisfies (C1), (C2) and is -entering, also satisfies (C1), (C2), thus the map is a representation map for . So, ample classes would admit representation maps, if for any ample classes , any representation map of extends to a -entering representation map of .
6.5. Representation Maps as ISRs
Our next result formulates the construction of representation maps for ample classes as an instance of the Independent System of Representatives problem. A system consisting of a graph and a partition of its vertex set is called an ISR-system. An independent set in of the form , where for each , is called an an Independent System of Representatives, or ISR for short [2].
Consider an ample class . We build an ISR-system as follows. For each concept and each set such that belongs to a -cube of , there is a vertex in . For each concept , set . Finally, is defined as follows: there is an edge between two vertices , in if are distinct vertices belonging to a common cube such that .
Proposition 6.16**.**
There is an ISR for if and only if admits a representation map.
Proof.
Assume first that is a representation map for . We show that is an ISR for . By (C1), for every , belongs to an -cube of and thus . Moreover, if is not an independent set of , there exist two concepts in a cube such that , contradicting (C2).
Conversely, if is an ISR for , then the map defined by is a representation map. Indeed, (C1) is satisfied by the definition of the vertices of , (C2) is satisfied by the definition of the edges of and since is an independent set of . ∎
Acknowledgements
We are very grateful to the referees of this submission and to some of the referees of previous versions of this paper for useful remarks that helped improving the presentation of this work. We are also grateful to Olivier Bousquet for several insightful discussions. The research of J. C. and V. C. was supported by the ANR project DISTANCIA (ANR-17-CE40-0015). S. M. is a Robert J. Shillman Fellow and was supported by the ISF, grant no. 1225/20, by an Azrieli Faculty Fellowship, and by BSF grant 2018385; part of this project was carried while S.M. was at the Institute for Advanced Study and was supported by the NSF grant CCF-1412958. The research of M. W. was partially supported by the NSF grant IIS-1619271.
Appendix A Two representation maps for Hall’s concept class
We give two representation maps for Hall’s concept class given in Figure 3. The representation map presented in Table 1 was found by constructing Boolean clauses for a matching between the concepts and the dimension sets of size that satisfies the non-clashing condition. A satisfying assignment was then found with the open source MiniSat solver. The representation map presented in Table 2 was found by following the steps of the proof of Theorem 5.1 by recursing on dimensions 12, 11, In both cases, the representation map , of is described as follows: we list all the 299 concepts as bit vectors and for each concept , we underline the bits of (a subset of size from ).
The USOs corresponding to these two representation maps are given in Figures 6 and 7; they were obtained by using Sage and Graphviz. In both cases, the non-trivial strongly connected components are represented by the blue boxes. Except from the arcs within these strongly connected components, the other arcs are downward arcs.
The USO from Figure 6 has a unique non-trivial strongly connected component (NSCC) while the one from Figure 7 has two such components and . Since those components are small (each of them contains concepts), these USOs are not “far” from being acyclic and is (not “far” from being) a minimal corner-free example.
Given a concept class and an orientation of the edges of , we denote by the resulting directed graph. From the following proposition, we deduce that any representation map for obtained by the algorithm of Theorem 5.1 contains at least two NSCCs. In particular, this shows that the representation map cannot be obtained in such a way.
Proposition A.1**.**
For any maximum class without corners, any representation map of computed by Theorem 5.1, and the USO corresponding to , the directed graph contains at least two non-trivial strongly connected components.
Proof.
We prove the result by induction on the dimension of . If is of dimension at most , then by Proposition 4.11 contains a corner and we are done.
Now suppose that the dimension of is at least 3. Recall that the first step is to contract along one of the coordinates (to construct , we contracted along coordinate , see Figure 8) and to compute recursively a USO (and the corresponding representation map ) of . Then we extend to a USO (and find the corresponding representation map ) of . Finally, we obtain from by keeping the orientation of all edges that do not correspond to coordinate and by orienting all -edges from to .
Observe that any directed cycle of corresponds either to a cycle of or to a directed cycle of a connected component of . If does not contain a corner, then by induction hypothesis, contains at least two NSCCs and thus there are two NSCCs in and thus in and we are done.
Suppose now that contains a corner. Consider a connected component of . If does not contain any directed cycle, then the concept class is acyclic and it contains a source . By (C1) applied to and , we conclude that is a corner of . Consequently, each connected component of contains at least one NSCC.
If is not a separator of , i.e., if is not a separator of , then without loss of generality, we can assume that there is no edge from to . Consider any corner of that is contained in a unique maximal cube of . In there exists a unique cube containing such that . Since has no neighbor outside the carrier , is the unique maximal cube containing and thus is a corner of , a contradiction.
We can thus assume that is a separator of and thus contains at least two connected components, and by the previous assertion, each of them contains at least one NSCC and we are done. ∎
A strongly connected component (SCC) of a directed graph is called a source-component if there is no arc from to with and .
In the directed graph corresponding to a representation map of an ample class , if a source-component is reduced to a single concept (i.e., is trivial), then is a corner of . Thus, one can view the source-components of as a generalization of corners. However, the definition of source-components depends on a given representation map and the corresponding orientation of , while the corners are defined in the undirected graph . In the following proposition, we prove that, similarly to corners, removing a source component from an ample class results into an ample class . Moreover, the restriction of the representation map to is still a representation map.
Proposition A.2**.**
Let be an ample class , be a representation map of , the USO defined by , and be the corresponding directed graph. Then for any source-component of , is an ample class and the restriction of to is a representation map of .
Proof.
Set and denote by the restriction of to . Since satisfies (C2) on , satisfies (C2) on . Now, we show that satisfies (C1) on . Pick any concept and let be the cube of containing and defined by . By (C1), is included in . If is also included in , then we are done. So, suppose that and pick a concept closest to . Let be the cube spanned by and . Since , is a cube of . By the definition of , is a source of and thus of . On the other hand, from the choice of all neighbors of in belong to . Since and is a source-component of , all these edges have as a source. Consequently, is a second source of , contrary to condition (C2) applied to . This shows that is included in , establishing that satisfies (C1) on .
Since is a bijective map from to , from (C1) applied to we conclude that is an injective map from to , yielding . Since by the sandwich lemma, we deduce that and thus is ample by Theorem 3.1(iv). Since the out-map of satisfies (C1) and (C2), is a representation map of . ∎
Corollary A.3**.**
* and are ample classes admitting corner peelings.*
Proof.
By Proposition A.2, the concept classes and are ample classes and the restrictions of to them are representation maps. Applying Proposition A.2 we conclude that is an ample class and the restriction of to is a representation map. Moreover, since all other NSCCs of are trivial, the orientations of the edges of graphs and defined by these representation maps are acyclic USOs. By Proposition 6.12, and admit corner peelings. ∎
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Karim Adiprasito and Bruno Benedetti. Collapsibility of CAT(0) spaces. Geom. Dedicata , 206:181–199, 2019. doi:10.1007/s 10711-019-00481-x . · doi ↗
- 2[2] Ron Aharoni, Eli Berger, and Ran Ziv. Independent systems of representatives in weighted graphs. Combinatorica , 27(3):253–267, 2007. doi:10.1007/s 00493-007-2086-y . · doi ↗
- 3[3] Richard P. Anstee, Lajos Rónyai, and Attila Sali. Shattering news. Graphs Combin. , 18(1):59–73, 2002. doi:10.1007/s 003730200003 . · doi ↗
- 4[4] Hans-Jürgen Bandelt and Victor Chepoi. Metric graph theory and geometry: a survey. In J. E. Goodman, J. Pach, and R. Pollack, editors, Surveys on Discrete and Computational Geometry: Twenty Years Later , volume 453 of Contemp. Math. , pages 49–86. Amer. Math. Soc., Providence, RI, 2008. doi:10.1090/conm/453/08795 . · doi ↗
- 5[5] Hans-Jürgen Bandelt, Victor Chepoi, Andreas W. M. Dress, and Jack H. Koolen. Combinatorics of lopsided sets. European J. Combin. , 27(5):669–689, 2006. doi:10.1016/j.ejc.2005.03.001 . · doi ↗
- 6[6] Hans-Jürgen Bandelt, Victor Chepoi, Andreas W. M. Dress, and Jack H. Koolen. Geometry of lopsided sets. 2012. Unpublished.
- 7[7] Hans-Jürgen Bandelt, Victor Chepoi, and Kolja Knauer. CO Ms: Complexes of oriented matroids. J. Combin. Theory Ser. A , 156:195–237, 2018. doi:10.1016/j.jcta.2018.01.002 . · doi ↗
- 8[8] Béla Bollobás and Andrew J. Radcliffe. Defect Sauer results. J. Combin. Theory Ser. A , 72(2):189–208, 1995. doi:10.1016/0097-3165(95)90060-8 . · doi ↗