Good Clusterings Have Large Volume

TL;DR

This paper explores the geometric properties of clusterings via polyhedral theory, revealing that high-quality clusterings correspond to vertices with large normal cones, which are also more likely to be produced by classical algorithms.

Contribution

It introduces a new quality measure based on the volume of normal cones and characterizes the structure of bounded-shape partition polytopes to analyze clustering stability and separation.

Findings

Good clusterings have large normal cones, indicating stability.

Classical algorithms tend to find clusterings with large normal cones.

Explicit descriptions of the polytope vertices enable stability analysis.

Abstract

The clustering of a data set is one of the core tasks in data analytics. Many clustering algorithms exhibit a strong contrast between a favorable performance in practice and bad theoretical worst-cases. Prime examples are least-squares assignments and the popular $k$-means algorithm. We are interested in this contrast and study it through polyhedral theory. Several popular clustering algorithms can be connected to finding a vertex of the so-called bounded-shape partition polytopes. The vertices correspond to clusterings with extraordinary separation properties, in particular allowing the construction of a separating power diagram, defined by its so-called sites, such that each cluster has its own cell. First, we quantitatively measure the space of all sites that allow construction of a separating power diagram for a clustering by the volume of the normal cone at the corresponding vertex. This gives rise to a new quality criterion for clusterings, and explains why good clusterings are also the most likely to be found by some classical algorithms. Second, we characterize the edges of the bounded-shape partition polytopes. Through this, we obtain an explicit description of the normal cones. This allows us to compute measures with respect to the new quality criterion, and even compute "most stable" sites, and thereby "most stable" power diagrams, for the separation of clusters. The hardness of these computations depends on the number of edges incident to a vertex, which may be exponential. However, the computational effort is rewarded with a wealth of information that can be gained from the results, which we highlight through some proof-of-concept computations.

Figures

Equations

$$\begin{array}[]{lcrclcl}s_{i}^{-}&\leq&\sum\limits_{j=1}^{n}y_{ij}&\leq&s_{i}^{+}&&(i\leq k)\\ &&\sum\limits_{i=1}^{k}y_{ij}&=&1&&(j\leq n)\\ &&y_{ij}&\geq&0&&(i\leq k,j\leq n).\end{array}$$

$$\sum _{i=1}^k\sum _{x\in C_i}{\Vert x-a_i\Vert }_2^2$$

$$P_i:=\{x\in {\mathbb{R}}^d|(a_j-a_i{)}^{T}x\le {\alpha }_i-{\alpha }_j\text{ for all }j\in [k]\setminus \{i\}\}$$

$$M:C_{i_1}\stackrel{x_{j_1}}{⟶}{C}_{i_2}\stackrel{x_{j_2}}{⟶}\cdots \stackrel{x_{j_t}}{⟶}{C}_{i_{t+1}}$$

$$d(a,a^{\prime }):=\inf \{L(\gamma )|\gamma :[0;1]\mapsto {\mathbb{R}}^{d\cdot k},\gamma (0)=a,\gamma (1)=a^{\prime },\gamma (t)\in {\mathbb{S}}^{d\cdot k}\forall t\in [0;1]\}.$$

$${\mu }_{\pm }(C):=\frac{\text{vol}(N_{{\mathcal{P}}^{\pm }}(w(C)))}{\text{vol}({\mathbb{R}}^{d\cdot k})}$$

$${\mu }_{=}(C):=\frac{\text{vol}(N_{{\mathcal{P}}^{=}}(w(C)))}{\text{vol}({\mathbb{R}}^{d\cdot k})}$$

$$a^{T}w(C)=\sum _{i=1}^k{\bar{a}}^T{\sigma }_i=\sum _{i=1}^k\sum _{x\in C_i}{\bar{a}}^{T}x=\sum _{j=1}^n{\bar{a}}^T{x}_j,$$

$${\mu }_{\star }(C)=\frac{\Gamma (\frac{d\cdot k}{2})}{2{\pi }^{\frac{d\cdot k}{2}}}\underset{\mathrm{bd}(B(N_{{\mathcal{P}}^{\star }}(w(C))))}{\int }d(z,a)da,$$

$${\tau }_{\pm }^p(a):=\max \{\delta >0|\bar{a}\in N_{{\mathcal{P}}^{\pm }}(w(C))\text{ for all }{\Vert a-\bar{a}\Vert }_p\le \delta \}.$$

$${\tau }_{=}^p(a):=\max \{\delta >0|\bar{a}\in N_{{\mathcal{P}}^{=}}(w(C))\text{ for all }{\Vert a-\bar{a}\Vert }_p\le \delta \}$$

$$\begin{array}[]{rll}\min&\left\|z\right\|_{2}^{2}&\\ \text{s.t.}&v_{j}^{T}z\leq\gamma^{p}_{j}&\,\forall\,j\in[t],\\ &z\in\mathbb{R}^{d\cdot k}&\end{array}$$

$$\begin{array}[]{rl}\min&v_{j}^{T}z\\ \text{s.t.}&z\in\mathbb{B}^{p}_{1}(0),\\ &z\in\mathbb{R}^{d\cdot k}\end{array}$$

$${\Vert \tilde{z}-z^{(p)}\Vert }_p=(\sum _{i=1}^k{\Vert {\tilde{z}}_i-z_i\Vert }_p^p{)}^{\frac{1}{p}}\le (\sum _{i=1}^k{\delta }^p{)}^{\frac{1}{p}}=\delta \cdot k^{\frac{1}{p}}<(\frac{1}{k}{)}^{\frac{1}{p}}\cdot k^{\frac{1}{p}}=1.$$

$${\Vert \tilde{z}-z^{(p)}\Vert }_{\infty }=\max \{{\Vert {\tilde{z}}_i-z_i\Vert }_{\infty }|i\in [k]\}\le \delta <1.$$

$$\begin{array}[]{rll}\min&\left\|z\right\|_{2}^{2}&\\ \text{s.t.}&v_{j}^{T}z\leq-\left\|v_{j}\right\|_{2}&\,\forall\,j\in[t],\\ &z\in\mathbb{R}^{d\cdot k}.&\end{array}$$

$$\max \{d(z,a)|a\in \gamma ,a\in B(N_{{\mathcal{P}}^{\pm }}(w(C)))\}=d(z,\bar{a})>0$$

$$\text{vol}(N_{{\mathcal{P}}^{\pm }}(w(C)))=\underset{\mathrm{bd}(B(N_{{\mathcal{P}}^{\pm }}w(C))))}{\int }d(z,a)da,$$

$$a^{T}w(\bar{C})=a^T(w(C^{\prime })+w(M^{-1}))=a^{T}w(C^{\prime })-a^{T}w(M)>a^{T}w(C^{\prime }).$$

$$M_1:$$

$$M_2:$$

$$x_{i_{l-1}}-x_{i_l},x_{i_{l-1}}-x_{i_l}^{\prime },$$

$$M_1:$$

$$M_2:$$

$$x_{j_t}-x_{j_1},x_{j_t}^{\prime }-x_{j_1}^{\prime },x_{j_t}^{\prime }-x_{j_1},x_{j_t}-x_{j_1}^{\prime },$$

$$\begin{array}[]{rll}\min&\left\|z\right\|_{2}^{2}&\\ \text{s.t.}&z\in N_{\mathcal{P}^{\pm}}(w(C)),&\\ &dist(z,F_{j})_{p}\geq 1&\,\forall\,j\in[t],\\ &z\in\mathbb{R}^{d\cdot k}.&\end{array}$$

$${\tau }_{\pm }^p(z)=\frac{{\delta }_z}{{\Vert z\Vert }_2}=\frac{1}{\frac{{\Vert z\Vert }_2}{{\delta }_z}}\le \frac{1}{{\Vert z^{(p)}\Vert }_2}={\tau }_{\pm }^p(z^{(p)}).$$

$$dist(z,\mathrm{lin}(F_j){)}_p\ge 1⟺z\in H_{(v_j,{\gamma }_j^p)}^{\le }\cup H_{(v_j,-{\gamma }_j^p)}^{\ge }.$$

$$z={\nu }_0^j{v}_j+\sum _{l=1}^{d\cdot k-1}{\nu }_l^j{f}_l^j\text{ with }{\nu }_0^j,\dots ,{\nu }_{d\cdot k-1}^j\in \mathbb{R}.$$

$$N_{{\mathcal{P}}^{\pm }}(w(C))\cap \{z\in {\mathbb{R}}^{d\cdot k}|dist(z,F_j{)}_p\ge 1\}=N_{{\mathcal{P}}^{\pm }}(w(C))\cap H_{(v_j,{\gamma }_j^p)}^{\le }$$

$$N_{{\mathcal{P}}^{\pm }}(w(C))=\bigcap _{j=1}^t{H}_{(v_j,0)}^{\le },$$

Full text

First, we quantitatively measure the space of all sites that allow construction of a separating power diagram for a clustering by the volume of the normal cone at the corresponding vertex. This gives rise to a new quality criterion for clusterings, and explains why good clusterings are also the most likely to be found by some classical algorithms. Second, we characterize the edges of the bounded-shape partition polytopes. Through this, we obtain an explicit description of the normal cones. This allows us to compute measures with respect to the new quality criterion, and even compute “most stable” sites, and thereby “most stable” power diagrams, for the separation of clusters. The hardness of these computations depends on the number of edges incident to a vertex, which may be exponential. However, the computational effort is rewarded with a wealth of information that can be gained from the results, which we highlight through some proof-of-concept computations.

1 Introduction

Informed decision-making based on large data sets is one of the key challenges in Operations Research. We are interested in one of the fundamental tasks in data analytics, the clustering of a data set into disjoint clusters. Data is often represented as a finite set $X\subset\mathbb{R}^{d}$ in $d$-dimensional Euclidean space. A clustering $C=(C_{1},\dots,C_{k})$ then is a partition of the set $X$ into disjoint clusters $C_{i}\subset X$, such that $\bigcup\limits_{i=1}^{k}C_{i}=X$ and $C_{i}\cap C_{j}=\emptyset$.

There is a wealth of literature on clusterings methods. We refer to three surveys by [9, 25, 37]. For many clustering algorithms, there is a strong contrast between an extremely favorable performance in practice and the lack of provable worst-case guarantees in theory. A prime example is the popular $k$-means algorithm proposed by [27, 28]. In practice, it typically terminates in just a few iterations and produces human-interpretable results. In a theoretical worst-case, it may take exponentially many iterations even for two-dimensional data (see [35]), and the resulting clustering may not capture the structure of the underlying data. In the present paper, we use methods of polyhedral theory to better understand this stunning discrepancy.

The studies of polyhedra have been a popular approach for applications in Operations Research. There are many cases where the combinatorial properties of these polyhedra revealed deeper insight into the underlying applications, c.f. [2, 13, 15, 16, 21, 22, 26, 31, 33]. For an introduction to polyhedral theory, we recommend [30, 32, 38]. Further, we refer the reader to the book review by [34] of the classical textbook Lectures on Polytopes by [38] for an in-depth account of polyhedral theory and its applications.

The so-called assignment polytopes are closely related to our setting and have been studied well, c.f. [6, 19, 20]. It is possible to represent the partition of data points $X=\{x_{1},\dots,x_{n}\}\subset\mathbb{R}^{d}$ into clusters $C_{1},\dots,C_{k}$ by using decision variables $y_{ij}$ to indicate whether data point $x_{j}$ is assigned to cluster $C_{i}$ ($y_{ij}=1$) or not ($y_{ij}=0$). Further, many applications specify lower bounds $s_{i}^{-}$ and upper bounds $s_{i}^{+}$ on the number of points that may be assigned to cluster $C_{i}$. This gives rise to a simple set of linear constraints that describes all clusterings:

[TABLE]

The first set of constraints makes sure the prescribed cluster size bounds are respected, the second set of constraints guarantees that each data point is assigned to a cluster. With the relaxed constraints $y_{ij}\geq 0$, we obtain a polytope $P$. The coefficient matrix of the constraints is totally unimodular, and the vector on the right-hand side is integral, so the vertices of this polytope are $0,1$-vectors, i.e. all $y_{ij}$ satisfy $y_{ij}\in\{0,1\}$. Each vertex describes a clustering, and vice versa.

To identify “good” clusterings, we study a projection of $P$ that includes information on the locations of the data points $X$ in $\mathbb{R}^{d}$. This projection, the so-called bounded-shape partition polytope, was first introduced in [7, 24]. (For a formal definition, see Section $2.3$.) The vertices of bounded-shape partition polytopes exhibit several favorable properties, c.f. [7], such as being a minimizer of the least-squares functional among all clusterings of the same cluster sizes. In particular, these vertices have strong separation properties. They allow for the construction of a separating power diagram, a generalized Voronoi diagram in $\mathbb{R}^{d}$ with one polyhedral cell for each cluster. In machine learning and other parts of Operations Research, this separation property is sometimes called piecewise-linear separability, c.f. [8]. Power diagrams are defined by a set of sites, $d$-dimensional vectors (one for each cell) which can be seen as the “centers” of the cells. See Figure 1 for a small example and Section 2.1 for a formal defintion.

Contributions and Outline

In Section $2$, we introduce some notation and review related work. We then use the known vertex characterization of [7] as a starting point for the new contributions in this paper. We briefly outline these contributions in the following paragraphs. Section $3$ is a complete and self-contained presentation of our results. Section $4$ contains the necessary proofs. We conclude our discussion with some final remarks in Section $5$. Parts of this work are based on the first author’s Ph.D. dissertation ([11]) and the second author’s M.Sc. thesis ([23]).

**Section 3.1. ** First, we devise a new measure for the quality of a clustering. We call it the volume of a clustering, since it is a measure for the volume of the normal cone of the vertex encoding this particular clustering in the bounded-shape and single-shape partition polytopes. (The volume of a cone is the standard (finite) Lebesgue measure of the cone intersected with the unit sphere.) This measure is quite different from the stability measures used in the literature, such as least-squares functionals and margins (see [36]). We will exhibit why a large volume indicates a clustering of high quality and distinguish it from the classical measures. This provides an informal explanation why many clustering algorithms work well in practice: For example, the computation of a least-squares assignment for fixed cluster sizes is in one-to-one correspondence to linear optimization over a single-shape partition polytope, see [11]. Further, the $k$-means algorithm can be interpreted as the repeated computation of least-squares assignments with changing sites in each iteration. When choosing random sites, the chance to find a clustering is directly correlated with its volume. This means that clusterings of large volume are found most often, so the best clusterings are the most likely to be found.

**Section 3.2. ** Second, we devise an explicit representation of the normal cones of vertices of the bounded-shape partition polytope. This is a challenging task, as there is no explicit description of the facets of the polytope. In a generalization of results by [17], we characterize the edges of all bounded-shape partition polytopes. They correspond to so-called movements or cyclical movements of items between clusters. This characterization enables us to construct the normal cone of a vertex explicitely and to investigate its structure. By this, we can identify convex areas which contain a representative site for all power diagrams inducing this clustering. We provide some proof-of-concept computations and a running example, in which we compare clusterings of different volumes.

**Section 3.3. ** Finally, we introduce a new stability criterion for sites for a clustering and provide an algorithm for the computation of optimal sites in the sense of this stability criterion. These sites are maximally stable with respect to perturbation, i.e. all sites can be perturbed in any direction with a largest possible amount without changing the clustering. We use a classical approach from computational geometry to find such sites: We roll a $p$-norm unit ball into the normal cone and compute where it gets “stuck”. The center of the ball gives the desired sites. This computation is readily expressed as a mathematical program. Hardness of the computation comes from the fact that there can be exponentially many edges. Of course, this hardness is not surprising – there are related problems, like the $k$-means problem, for which the complexity of finding a global optimum (globally optimal sites) is known to be $\mathbb{NP}$-hard, even for $k=2$ (see [1]) or for data in the Euclidean plane, c.f. [29].

2 Preliminaries

We begin with some standard notation. Let $A\subseteq\mathbb{R}^{d}$. Then $\operatorname{lin}(A)$ denotes the minimal linear subspace containing $A$. If $A$ is convex, we call $\operatorname{int}(A)$ the interior and $\operatorname{bd}(A)$ the boundary of $A$.

2.1 Clusterings, Least-Squares Assignments and Power Diagrams

Let throughout this paper $n,d,k\in\mathbb{N}:=\{1,2,\dots\}$ be fixed. Let $X:=\{x_{1},\dots,x_{n}\}\subseteq\mathbb{R}^{d}$ be a set of $n$ distinct non-zero data points and for $m\in\mathbb{N}$ define $[m]:=\{1,\dots,m\}$. We call a partition $C:=(C_{1},\dots,C_{k})$ of $X$ a clustering and call $\left|C\right|:=(\left|C_{1}\right|,\dots,\left|C_{k}\right|)$ its shape. For $i\in[k]$, we call $C_{i}$ the $i$-th cluster of $C$ and $\left|C_{i}\right|$ its size. Let $s^{-}:=(s^{-}_{1},\dots,s^{-}_{k})$, $s^{+}:=(s_{1}^{+},\dots,s_{k}^{+})\in\mathbb{N}^{k}$ such that $0\leq s^{-}_{i}\leq s^{+}_{i}\leq n$ for all $i\in[k]$ be the lower and upper bounds on the cluster sizes.

A clustering $C$ is said to be feasible if it satisfies $s^{-}\leq\left|C\right|\leq s^{+}$ componentwisely. We will only consider feasible clusterings in this paper. $C$ is called separable if all pairs of clusters are linearly separable, i.e. for all $i,j\in[k]$, $i\not=j$ there is $a_{ij}\in\mathbb{R}^{d}$ and $\gamma_{ij}\in\mathbb{R}$ such that $a_{ij}^{T}x\leq\gamma_{ij}\leq a_{ij}^{T}y$ for all $x\in C_{i}$, $y\in C_{j}$. The hyperplane separating the clusters is denoted by $H_{(a_{ij},\gamma_{ij})}:=\{x\in\mathbb{R}^{d}\,|\,a_{ij}^{T}x=\gamma_{ij}\}$. Analogously, we define $H^{\leq}_{(a_{ij},\gamma_{ij})}$ and $H^{\geq}_{(a_{ij},\gamma_{ij})}$ to be the respective half-spaces. A constrained least-squares assignment (LSA) for a given set of sites $a_{1},\dots,a_{k}\in\mathbb{R}^{d}$ is a clustering $C=(C_{1},\dots,C_{k})$ minimizing

[TABLE]

over all clusterings with the same shape as $C$. If $C$ minimizes (1) over all feasible clusterings, we say $C$ is a (general) LSA. We call $a=(a_{1}^{T},\dots,a_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ the site vector.

Recall that power diagrams are a generalization of the well-known Voronoi diagrams (see [3]). There are several equivalent definitions of power diagrams. We briefly recall the definition that is best for our purposes (see [12]). Let $a:=(a_{1}^{T},\dots,a_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ be a site vector with distinct sites $a_{1},\dots,a_{k}\in\mathbb{R}^{d}$ and let $\alpha_{1},\dots,\alpha_{k}\in\mathbb{R}$. For $i\in[k]$, we call

[TABLE]

the $i$-th cell of the power diagram $(P_{1},\dots,P_{k})$.

[4] showed the following connection between constrained LSAs and power diagrams: If $C$ is a constrained LSA to the site vector $a$, then there is a power diagram with site vector $a$ satisfying $C_{i}\subseteq\operatorname{int}(P_{i})$ for all $i\in[k]$. On the other hand, if a power diagram $(P_{1},\dots,P_{k})$ with site vector $a$ satisfies $C_{i}\subseteq P_{i}$ for all $i\in[k]$, then $C$ is a constrained LSA to the site vector $a$. If $C_{i}\subseteq P_{i}$ for all $i\in[k]$, we say the power diagram induces the clustering and call it a separating power diagram.

2.2 Movements Between Clusterings

In order to compare two clusterings $C:=(C_{1},\dots,C_{k})$, $C^{\prime}:=(C^{\prime}_{1},\dots,C^{\prime}_{k})$, we define the clustering difference graph (CDG) to be the labeled directed multigraph $CDG(C,C^{\prime}):=(V,E)$ with node set $V:=[k]$ and edge set $E$ constructed as follows: For each $x_{j}\in C_{i}\cap C^{\prime}_{l}$ with distinct $i,l\in[k]$, there is an edge $(i,l)\in E$ with label $x_{j}$. W.l.o.g. we delete isolated nodes in the CDG, since these would correspond to clusters that are identical in $C$ and $C^{\prime}$. We can derive $C^{\prime}$ from $C$ by applying operations corresponding to the edges of $CDG(C,C^{\prime})$.

Let $(i_{1},i_{2})-(i_{2},i_{3})-\cdots-(i_{t},i_{t+1})$ be an edge path in $CDG(C,C^{\prime})$ with labels $x_{j_{1}},\dots,x_{j_{t}}$. Applying the movement

[TABLE]

to $C$ means deriving the clustering $\bar{C}=(\bar{C}_{1},\dots,\bar{C}_{k})$ by setting $\bar{C}_{i_{l}}:=(C_{i_{l}}\setminus\{x_{j_{l}}\})\cup\{x_{j_{l-1}}\}$ for all $l\in\{2,\dots,t\}$, $\bar{C}_{i_{1}}:=C_{i_{1}}\setminus\{x_{j_{1}}\}$, $\bar{C}_{i_{t+1}}:=C_{i_{t+1}}\cup\{x_{j_{t}}\}$ and $\bar{C}_{r}:=C_{r}$ for all $r\in[k]\setminus\{i_{1},\dots,i_{t+1}\}$.

If $i_{t+1}=i_{1}$, i.e. in case of a cycle, we speak of a cyclical movement. We then obtain $\bar{C}_{i_{1}}=\bar{C}_{i_{t+1}}:=(C_{i_{1}}\setminus\{x_{j_{1}}\})\cup\{x_{j_{t}}\}$ and all cluster sizes remain the same. The inverse (cyclical) movement $M^{-1}$ is defined via the corresponding path (cycle) $(i_{t+1},i_{t})-(i_{t},i_{t-1})-\cdots-(i_{2},i_{1})$ in $CDG(C^{\prime},C)$. Clearly, one can obtain any clustering from any other clustering by (greedily) decomposing their clustering difference graph into paths and cycles and applying the corresponding (cyclical) movements to $C$. If $\left|C\right|=\left|C^{\prime}\right|$, then $CDG(C,C^{\prime})$ decomposes into cycles, i.e. cyclical movements suffice to transform $C$ into $C^{\prime}$, c.f. [11].

Figures 2 and 3 depict a clustering of twelve data points in $\mathbb{R}^{2}$ with $C_{1}$ black, $C_{2}$ blue, $C_{3}$ red, as well as two clusterings which can be derived from it by applying a movement (Figure 2) and a cyclical movement (Figure 3), respectively.

2.3 Bounded-Shape and Single-Shape Partition Polytopes

The polytope we are studying was introduced in [7] and [24]:

For a clustering $C=(C_{1},\dots,C_{k})$ and $i\in[k]$, let $\sigma_{i}:=\sum\limits_{x\in C_{i}}x\in\mathbb{R}^{d}$. The clustering vector of $C$ is $w(C):=(\sigma_{1}^{T},\dots,\sigma_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$. Then $\mathcal{P}^{\pm}(X,k,s^{-},s^{+}):=\operatorname{conv}(\{w(C)\ |\ C$ is feasible$\})$ is called the bounded-shape partition polytope. If $s=s^{-}=s^{+}$, then we call $\mathcal{P}^{=}_{s}=\mathcal{P}^{\pm}(X,k,s,s)$ the single-shape partition polytope. Another interesting special case is the all-shape partition polytope $\mathcal{P}$ investigated by [17] which is obtained by choosing $s^{-}=(0,\dots,0)$ and $s^{+}=(n,\dots,n)$. When the bounds are clear from the context, we use the simpler notation $\mathcal{P}^{\pm}=\mathcal{P}^{\pm}(X,k,s^{-},s^{+})$ and $\mathcal{P}^{=}=\mathcal{P}^{=}_{s}$.

Note that $\mathcal{P}^{\pm}$ is a projection of the generalized assignment polytope investigated in [20]. We want to stress that, since these polytopes are defined as a convex hull, we do not have explicit information on (facet-defining) valid inequalities. This will play an important role in our discussion, as we have to study the edge structure of the polytopes (which may have exponential size) in order to construct the normal cones of its vertices. We use the notation $N_{P}(v):=\{a\in\mathbb{R}^{d}\ |\ a^{T}v\geq a^{T}x\ \forall\,x\in P\}$ for the normal cone of $v$ in a polytope $P$.

We observe the following connection between single-shape and bounded-shape partition polytopes.

Lemma 1

$\mathcal{P}^{\pm}(X,k,s^{-},s^{+})=\operatorname{conv}(\bigcup\limits_{s^{-}\leq s\leq s^{+}}\mathcal{P}^{=}_{s})$.

Proof 1

Proof. Let $U:=\bigcup\limits_{s^{-}\leq s\leq s^{+}}\mathcal{P}^{=}_{s}$ and let $w(C)\in\mathcal{P}^{\pm}$ be a clustering vector. Then $w(C)\in\mathcal{P}^{=}_{\left|C\right|}\subseteq U$. By definition of $\mathcal{P}^{\pm}$, we thus obtain $\mathcal{P}^{\pm}\subseteq\operatorname{conv}(U)$. On the other hand, clearly $\mathcal{P}^{=}_{s}\subseteq\mathcal{P}^{\pm}$ for all $s^{-}\leq s\leq s^{+}$ and thus $U\subseteq\mathcal{P}^{\pm}$. Taking the convex hull on both sides yields $\operatorname{conv}(U)\subseteq\operatorname{conv}(\mathcal{P}^{\pm})=\mathcal{P}^{\pm}$ where the last equality is due to the convexity of $\mathcal{P}^{\pm}$. \qed

Note that Lemma 1 implies that every vertex $w(C)$ of $\mathcal{P}^{\pm}$ is also a vertex of $\mathcal{P}^{=}_{\left|C\right|}$ and that $N_{\mathcal{P}^{\pm}}(w(C))\subseteq N_{\mathcal{P}^{=}}(w(C))$ for every clustering vector $w(C)$. [7] gave a first characterization of the vertices of $\mathcal{P}^{\pm}$.

Proposition 2** (Barnes, Hoffman, Rothblum 1992)**

The clustering vector $w(C)$ of a clustering $C$ is a vertex of $\mathcal{P}^{\pm}$ if and only if there are $a:=(a^{T}_{1},\dots,a_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ and $\alpha_{1},\dots,\alpha_{k}\in\mathbb{R}$ satisfying the following statements.

If $\left|C_{i}\right|>s^{-}_{i}$ for $i\in[k]$, then $\alpha_{i}\leq 0$. 2. 2.

If $\left|C_{i}\right|<s^{+}_{i}$ for $i\in[k]$, then $\alpha_{i}\geq 0$. 3. 3.

If $x_{l}\in C_{i}$ for $l\in[n],i\in[k]$, then for all $j\in[k]\setminus\{i\}$ it holds $(a_{j}-a_{i})^{T}x_{l}<\alpha_{i}-\alpha_{j}$.

A proof is given in [7] and, with more technical detail, in [24]. We call a clustering corresponding to a vertex of $\mathcal{P}^{\pm}$ a vertex clustering. Condition $3$ states linear separability of the clusters with separation directions $a_{ij}:=a_{j}-a_{i}\in\mathbb{R}^{d}$ and right-hand sides $\gamma_{ij}:=\alpha_{i}-\alpha_{j}$ for all $i,j\in[k]$. This implies the existence of a separating power diagram. This property can alternatively be derived from the observation that computation of an optimal constrained LSA corresponds precisly to linear optimization over the corresponding single-shape partition polytope $\mathcal{P}^{=}$ (see [11]).

If the scalars $\alpha_{1},\dots,\alpha_{k}$ additionally satisfy conditions $1$ and $2$, then the resulting separation fulfills some additional properties. For example, if two clusters satisfy $s^{-}<\left|C_{i}\right|,\left|C_{j}\right|<s^{+}$, then $C_{i}$ and $C_{j}$ are “[math]-separable” (see [5]), in particular they can be separated by a hyperplane containing the origin. Any vector $a\in\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))$ can be chosen to construct suitable $\alpha_{1},\dots,\alpha_{k}$ such that the properties of Proposition 2 are satisfied, c.f. [7].

This gives the following corollary (see [11, 23]).

Corollary 3

Let $C:=(C_{1},\dots,C_{k})$ such that $w(C)$ is a vertex of $\mathcal{P}^{\pm}$ and let $a:=(a_{1}^{T},\dots,a_{k}^{T})^{T}\in N_{\mathcal{P}^{\pm}}(w(C))\subseteq\mathbb{R}^{d\cdot k}$. Then there is a separating power diagram $(P_{1},\dots,P_{k})$ with site vector $a\in\mathbb{R}^{d\cdot k}$ such that $C_{i}\subseteq P_{i}$ for all $i\in[k]$. If $a\in\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))$, then $C_{i}\subseteq\operatorname{int}(P_{i})$ for all $i\in[k]$. If $a\in\operatorname{bd}(N_{\mathcal{P}^{\pm}}(w(C))\subseteq\mathbb{R}^{d\cdot k}$, then there is an index $i\in[k]$ such that $C_{i}\cap\operatorname{bd}(P_{i})\not=\emptyset$.

The statement also holds for the single-shape partition polytope $\mathcal{P}^{=}$ by replacing “$\pm$” by “$=$”. Further, the normal cone of a vertex clustering of the single-shape partition polytope encodes exactly all site vectors that allow a separating power diagram (see [11]). Clearly, if $a$ is in the normal cone, then so is $\lambda a$ for every $\lambda>0$. Thus $a$ and $\lambda a$ yield the same constrained LSA. This was first proven by [4], in a different notation.

Proposition 4** (Aurenhammer, Hoffmann, Aronov 1998)**

Let $a\in\mathbb{R}^{d\cdot k}$ be a site vector of a constrained LSA. For all $\lambda>0$, the site vectors $\lambda a$ yield the same constrained LSA.

Before we turn to our main results, we would like to mention two tools that will make our arguments easier. First, recall our assumption that the zero vector is not contained in $X$. This is no restriction, since the overall structure of a data set is not changed when translating the whole set by the same vector. Second, we can interpret any movement (recall Section 2.2) as a translation of the corresponding clustering vector. Let $C,C^{\prime}$ such that $CDG(C,C^{\prime})$ is a single path or cycle corresponding to a (cyclical) movement $M$. Then the difference of the clustering vectors $w(M):=w(C^{\prime})-w(C)$ is called the vector of the movement $M$. Note that the vector of the inverse movement is given by $w(M^{-1})=-w(M)$.

3 Main results

We begin each section with a brief overview.

3.1 Volume of Clusterings

Overview.* Maximizing the linear objective vector $a=(a_{1}^{T},\dots,a_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ over the bounded-shape partition polytope yields a vertex clustering. The sites $a_{i}\in\mathbb{R}^{d}$ allow the construction of a separating power diagram for the clusters, c.f. [7, 11, 24]. Moreover, site vectors are invariant under scaling (Proposition 4). Combining the properties listed in Section 2.3 enables us to quantitatively measure the space of all sites that allow the construction of a separating power diagram inducing a given clustering by the volume of its normal cone. This gives rise to a quality measure that we call the volume of a clustering.*

Instead of considering each site vector $a\in\mathbb{R}^{d\cdot k}$ individually, we consider its equivalence class $[a]:=\{\lambda a\ |\ \lambda>0\}$ and choose the unit vector $\frac{1}{\left\|a\right\|_{2}}a$ as a representative. This allows us to introduce a notion of “distance of sites”. Let $L(\gamma)$ be the length of a curve $\gamma$ and $\mathbb{S}^{d\cdot k}:=\{x\in\mathbb{R}^{d\cdot k}\ |\ \left\|x\right\|_{2}=1\}$ be the Euclidean unit sphere.

Definition 5** (Distance of Sites)**

Let $a,a^{\prime}\in\mathbb{S}^{d\cdot k}$ be two site vectors. The distance of the equivalence classes $[a]$ and $[a^{\prime}]$ is defined as the distance of the site vectors on the unit sphere, i.e.

[TABLE]

Note that $d:\mathbb{S}^{d\cdot k}\times\mathbb{S}^{d\cdot k}\mapsto\mathbb{R}$ is a metric, takes values between [math] and $\pi$, and that an infimum always exists, because $L(\gamma)\geq 0$ for all $\gamma:[0;1]\mapsto\mathbb{R}^{d\cdot k}$.

If a vertex clustering $C$ of $\mathcal{P}^{\pm}$ or $\mathcal{P}^{=}$ has a large normal cone, then a randomly chosen site vector is likely to lie in its cone. By the previous observations, the chosen site vector defines a separating power diagram inducing $C$. We are interested in measuring the volume of the normal cones of the bounded-shape and single-shape partition polytopes in order to characterize “good” clusterings.

We follow the notation of **[10]** who used the volume of normal cones for the studies of combinatorial diameters. For a cone $K\subseteq\mathbb{R}^{d\cdot k}$, we call $B(K):=K\cap\mathbb{S}^{d\cdot k}$ the base of $K$. The volume of $K$ is then defined as the $(d\cdot k-1)$-dimensional volume of $B(K)$ and is denoted by $\operatorname{vol}(K)$. A set $A\subseteq\mathbb{S}^{d\cdot k}$ is called spherically convex, if for all $x,y\in A$ the geodesic $\gamma:[0;1]\mapsto\mathbb{S}^{d\cdot k}$ connecting $x$ and $y$ with $\gamma(0)=x$ and $\gamma(1)=y$ is contained in $A$. Recall that a geodesic is a curve on the sphere with shortest length. Note that the base of a cone is spherically convex itself. These notions allow us to formally introduce two variants of a new term which we call the volume of a clustering.

Definition 6** (Volume of a Clustering)**

Let $C$ be a feasible clustering and let $N_{\mathcal{P}^{\pm}}(w(C))$ and $N_{\mathcal{P}^{=}}(w(C))$ be its normal cones of $\mathcal{P}^{\pm}$ and $\mathcal{P}^{=}$, respectively. We define

[TABLE]

to be the BHR volume of $C$ (named after [7] who first studied this polytope) and

[TABLE]

to be the LSA volume of $C$.

The volume of a clustering puts the volume of the respective normal cones in relation to the volume of the whole space, so by definition $\mu_{\pm},\mu_{=}\in[0;1]$. The only difference is that the BHR volume compares the clustering to all feasible clusterings and the LSA volume compares it to all clusterings of the same shape. In the following, we use the simple wording “volume of a clustering” to refer to both variants when they behave analogously.

Note that vol$(\mathbb{R}^{d\cdot k})$* equals the area of the surface of $\mathbb{S}^{d\cdot k}$. In practice, permutations of the clusters/sites yield the same clustering and the respective normal cones have equal volume. For our purposes, it is more useful to consider each permutation as an individual clustering. However, if all cluster bounds are symmetric, then the volume defined above only takes values between 0 and $\frac{1}{k!}$, because it only takes one possible permutation into account. By Lemma 1, $\mu_{\pm}(C)\leq\mu_{=}(C)$ for all clusterings $C$.*

The volume of a clustering quantifies the “fraction” of all site vectors of separating power diagrams that induce the given clustering. This is quite a different concept than the two widely used quality measures for clusterings: the least-squares functional, which measures the quadratic Euclidean distance of the data points to their respective site, and the so-called margin, which measures the smallest Euclidean distance of data points to the nearest separating hyperplane (see **[12]**). Let us exhibit the difference between the LSA volume and the classical measures. (The BHR volume behaves analogously.)

Consider Figure 4, which depicts two similar data sets, where three “point clouds” of the same structure are at different distance to each other. The least-squares values of both clusterings are equal, whereas the right-hand clustering has a larger volume. This can be seen by noting that all power diagrams that induce the left-hand clustering also induce the right-hand one. Conversely, there are many power diagrams that induce the right-hand clustering, but not the left-hand one. This means that the set of site vectors that give the left-hand clustering is a strict subset of the set of site vectors that give the right-hand one. Informally, unlike a least-squares functional, the volume is able to take the distance between different clusters into account.

Now, consider Figure 5. The only difference between the data sets lies in the vertical scaling of the points in the top clusters and the horizontal scaling of the points in the bottom cluster. Both clusterings have equal margin, but the one on the right has a lower volume. It is easy to see that the separating hyperplanes on the right can be perturbed less due to the long, drawn-out clusters. This implies that the site vectors that define a separating power diagram inducing the right-hand clustering is a strict subset of the site vectors for the left-hand clustering. While the margin informally measures the ability to shift the separating hyperplanes closer to the clusters (without changing their directions), the volume measures the ability to perturb the normals of the separating hyperplanes.

Note that it does not matter whether we consider $\mathbb{R}^{d\cdot k}$ or the affine hull of the polytope in Definition 6, because all normal cones have a lineality space $\operatorname{lin}(\mathcal{P}^{\star}-v)^{\perp}$ with $v\in\mathcal{P}^{\star}$ for $\star\in\{\pm,=\}$, respectively. Since we consider the volume of a cone relative to the whole space, we can restrict to the affine hull of the polytopes (see **[23]** for further details).

In fact, one can show that $\mathcal{P}^{\pm}$ and $\mathcal{P}^{=}$ are contained in a $(d\cdot(k-1))$-dimensional affine subspace. Let $a=(\bar{a}^{T},\dots,\bar{a}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ with $\bar{a}\in\mathbb{R}^{d}$ and consider an arbitrary clustering vector $w(C)$. Then

[TABLE]

so $\mathcal{P}^{\star}\subseteq\{x\in\mathbb{R}^{d\cdot k}\ |\ (\bar{a}^{T},\dots,\bar{a}^{T})x=\sum\limits_{j=1}^{n}\bar{a}^{T}x_{j}\}$ for $\star\in\{\pm,=\}$. Thus the dimension is at most $d\cdot k-d=d\cdot(k-1)$.

We conclude the section by showing that the volume of a clustering is well-defined, and by providing an explicit algebraic description.

Theorem 7

The measures $\mu_{\pm}$ and $\mu_{=}$ are well-defined. For $\star\in\{\pm,=\}$, $C$ is a vertex clustering of $\mathcal{P}^{\star}$ if and only if $\mu_{\star}(C)>0$ and then

[TABLE]

with $z\in\operatorname{int}(N_{\mathcal{P}^{\star}}(w(C)))\cap\mathbb{S}^{d\cdot k}$ and $\Gamma$ being the Gamma function.

Note that the coefficient in (2) is the inverse of the area of the surface of $\mathbb{S}^{d\cdot k}$. We postpone the proof of this theorem to Section 4.1.

3.2 Site Vectors in the Normal Cone

Overview.* We derive an explicit description of the normal cones of the bounded-shape partition polytope via a characterization of the edges of the polytope. As a corollary, we also obtain a description for the normal cones of the single-shape partition polytope. Informally, two clusterings correspond to neighboring vertices of the polytope if they differ by only a single movement or cyclical movement. The explicit description of the normal cone allows a representation of the set of sites that define a separating power diagram for a vertex clustering in the form of $k$ convex areas in the space of the underlying data set.*

It is well-known that the edges incident to a vertex of a polytope are normal vectors to the facets of the normal cone of the vertex. Therefore, in order to obtain an explicit representation of the normal cone, we characterize the edges of the bounded-shape partition polytope. Our characterization generalizes previous results for the special case $d=1$, c.f. **[18]**.

The closest result in the literature is by **[17]**, who characterized the neighborhood of a vertex $w(C)$ of the all-shape partition polytope $\mathcal{P}$. They showed that edges incident to $w(C)$ correspond to movements of the form $C_{i}\stackrel{{\scriptstyle x}}{{\longrightarrow}}C_{j}$ or $C_{i}\stackrel{{\scriptstyle x}}{{\longrightarrow}}C_{j}\stackrel{{\scriptstyle\nu x}}{{\longrightarrow}}C_{i}$ with $\nu<0$ and $i,j\in[k]$, $i\not=j$. Moreover, they proved that, if $\operatorname{lin}(\{x\})\cap X=\{x\}$ for all $x\in X$ then all edges of $\mathcal{P}$ correspond to movements that move a single element from one cluster to another. The following theorem extends their result to general lower and upper bounds $s^{-}$ and $s^{+}$.

Theorem 8

Let $C:=(C_{1},\dots,C_{k})$, $C^{\prime}:=(C^{\prime}_{1},\dots,C^{\prime}_{k})$ be two clusterings such that $w(C)$ and $w(C^{\prime})$ are adjacent vertices of $\mathcal{P}^{\pm}$. Suppose that no three points in $X$ lie on a single line. Then $C$ and $C^{\prime}$ differ by a single (cyclical) movement or by two movements and there are distinct $i,j\in[k]$ such that both are of the form $C_{i}\rightarrow C_{j}$.

The different cases of Theorem 8 are depicted in Figure 6. Note that the case of two movements can only occur if all sites lie on a line. For the single-shape partition polytope, we obtain the following corollary.

Corollary 9

Let $C:=(C_{1},\dots,C_{k})$, $C^{\prime}:=(C^{\prime}_{1},\dots,C^{\prime}_{k})$ be two clusterings such that $w(C)$ and $w(C^{\prime})$ are adjacent vertices of $\mathcal{P}^{=}$. If no four points in $X$ lie on a single line, then $C$ and $C^{\prime}$ only differ by a single cyclical movement.

We dedicate Section 4.2 to the proof of this theorem and corollary. Figure 7 shows two vertex clusterings of $\mathcal{P}^{=}$ which are connected by an edge. The site vector of the plotted power diagram lies on the boundary of the normal cones of both vertices. As one can see, the data points on the boundary of the cells, which exist due to Corollary 3, move to the other cell.

Proof-of-Concept Computations

Figure 8 illustrates two clusterings of 27 data points in $\mathbb{R}^{2}$. These clusterings were computed by running the $k$-means algorithm 50 times with three random sites in the beginning. In every iteration the $k$-means algorithm computes a LSA to the current sites and updates each site as the arithmetic mean of the points in the cluster. This is repeated until the clustering does not change anymore. Note that the $k$-means algorithm is deterministic, but its result depends on the choice of the initial sites. Whereas the clustering on the left-hand side (except for permutation of the colors) was the output in 25 of the 50 runs, the one on the right-hand side was returned only once. Intuitively, one sees that the left-hand clustering captures the structure of the data better than the one on the right-hand side.

This fits with our quantitative measure: Both volumes of the clustering on the left-hand side are higher ($\mu_{\pm}\approx 0.0076$, $\mu_{=}\approx 0.041$) than the ones of the clustering on the right-hand side ($\mu_{\pm}<0.0001$, $\mu_{=}<0.01$). The volumes of the respective normal cones were computed with MATLAB using the function Volume Computation of Convex Bodies of **[14]** with an error tolerance of 0.001. The different volumes can also be verified by computing the edges of a respective normal cone and projecting the edges (which are (normalized) site vectors themselves) to their $d$-dimensional components for each of the $k$ sites. Each site of a site vector in the normal cone is located in the convex hull of the corresponding sites of the edges of the normal cone.

Figures 9 and 10 illustrate the three areas of the $2$-dimensional sites for the respective clusterings w.r.t. $\mathcal{P}$ and $\mathcal{P}^{=}$. Because of the invariance of power diagrams under scaling of sites (Proposition 4), we have to choose a fixed scaling of the $6$-dimensional site vectors to obtain a meaningful visualization. Here the site vectors are scaled to Euclidean norm 4.

One sees that the areas on the left-hand side are larger than the ones on the right-hand side, especially for the all-shape partition polytope (Figure 9). Note that these areas do not mean that one can choose three sites arbitrarily within the three areas and obtain an optimal site vector for the respective clustering. Instead, for any arbitrary site within one of these areas, there are sites in the respective other areas such that the corresponding site vector is inside the normal cone of the clustering. Further, these areas contain sites of a representative for all equivalence classes of site vectors in the normal cone. For the single-shape partition polytope, these areas depict representatives of all site vectors for which the respective clusterings are optimal constrained LSAs, see Figure 10.

3.3 Stability of Site Vectors

Overview.* We observed that clusterings with high volume are good for two reasons. They are likely to be computed for random sites and any site vector can be perturbed significantly without changing the clustering. Given $w(C)$ and its normal cone, we now characterize a most stable site vector inducing $C$. After introducing our notion of stability, which depends on the choice of a $p$-norm, we provide an optimization problem whose optimal solution gives us a site vector with the highest possible stability for this clustering. Moreover, we present how the optimal solutions for different $p$-norms are connected and how one can obtain an approximate solution for any $p$-norm by using the Euclidean norm.*

Definition 10** (Stability of a Site Vector)**

Let $p\in[1;\infty]$. For $a\in B(N_{\mathcal{P}^{\pm}}(w(C)))\subseteq\mathbb{S}^{d\cdot k}$ the BHR stability of the site vector $a$ w.r.t. $p$ is

[TABLE]

For $a\in B(N_{\mathcal{P}^{=}}(w(C)))\subseteq\mathbb{S}^{d\cdot k}$ we call

[TABLE]

the LSA stability of the site vector $a$ w.r.t. $p$.

The BHR and LSA stability, $\tau^{p}_{\pm}$ and $\tau^{p}_{=}$, measure how much we can perturb the site vector $a$ within the respective normal cone w.r.t. the $p$-norm without changing the induced clustering. Of special interest are $p=2,\infty$. Whereas the Euclidean norm weighs the perturbation of all sites equally, the infinity norm considers the highest possible perturbation of one individual site. We highlight the differences after stating our main theorem.

Note that the above definition can be extended to all site vectors by inserting the corresponding representative of the equivalence class into $\tau^{p}_{\star}$, $\star\in\{\pm,=\}$. Geometrically, a “most stable” site vector w.r.t $p$ can be described by dropping a $p$-norm unit ball into the normal cone with $0\in\mathbb{R}^{d\cdot k}$ as gravity center and computing where it gets stuck due to being blocked by the facets of the cone. The center of this unit ball then is a vector which lies “most centrally” within the normal cone. Figure 11 illustrates two examples of this approach.

Theorem 11

Let $p\in[1;\infty]$, $\star\in\{\pm,=\}$ and $w(C)\in\mathcal{P}^{\star}$ be a vertex with incident edges $v_{1},\dots,v_{t}$. Then the optimal solution of the following optimization problem yields a site vector inducing the clustering $C$ with highest possible stability w.r.t. $p$.

[TABLE]

with $\gamma^{p}_{1},\dots,\gamma^{p}_{t}$ being the optimal objective values of the problems

[TABLE]

for each $j\in[t]$. Here $\mathbb{B}^{p}_{\lambda}(b):=\{y\in\mathbb{R}^{d\cdot k}\ |\ \left\|y-b\right\|_{p}\leq\lambda\}$ denotes the closed $p$-norm ball with center $b\in\mathbb{R}^{d\cdot k}$ and radius $\lambda>0$.

Minimizing the squared Euclidean norm corresponds to dropping the $p$-norm unit ball into the cone with the origin as gravity center that attracts the ball. The constraints ensure that the ball remains inside the cone. We postpone a proof of the theorem to Section 4.3.

The stability of an optimal solution $z^{(p)}$ of (3) is $\tau^{p}_{\star}(z^{(p)})=\frac{1}{\left\|z^{(p)}\right\|_{2}}$. Problem (3) is a quadratic optimization problem with linear constraints and the auxiliary problems (4) are linear optimization problems over a convex set. The edges $v_{j}$ encode single (cyclical) movements for all $j\in[t]$ (Theorem 8 and Corollary 9). Note that there might be exponentially many edges and, thus, we might have to solve exponentially many auxiliary problems.

Problem (3) models the facets of the normal cone blocking the unit ball. We justify our approach that the ball is blocked by facets, rather than lower-dimensional faces of the cone, with a short example in Section 4.3.

Fix $p\in[1;\infty)$ and let $z^{(p)}:=(z_{1}^{T},\dots,z_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ be an optimal solution of (3). Then we can perturb one site, say $z_{1}\in\mathbb{R}^{d}$, within a $p$-norm ball with radius 1 without changing the clustering. For $0<\delta<k^{-\frac{1}{p}}$ and $\tilde{z}:=(\tilde{z}_{1}^{T},\dots,\tilde{z}_{k}^{T})^{T}\in\mathbb{R}^{d\cdot k}$ with $\tilde{z}_{i}\in\mathbb{B}^{p}_{\delta}(z_{i})\subseteq\mathbb{R}^{d}$ for all $i\in[k]$, we obtain

[TABLE]

Thus, $\tilde{z}\in\operatorname{int}(\mathbb{B}^{p}_{1}(z^{(p)}))\subseteq\operatorname{int}(N_{\mathcal{P}^{\star}}(w(C)))$, i.e. we can perturb each site within a $p$-norm ball with e.g. radius $\delta:=(k+1)^{-\frac{1}{p}}<k^{-\frac{1}{p}}$ without changing the clustering.

If $p=\infty$, we can even choose $0<\delta<1$, because then

[TABLE]

Next, we show how to obtain a feasible approximate solution of (3). It is well-known that for all $p,q\in[1;\infty]$ there is a positive constant $c_{p,q}>0$ only depending on $p,q$ and the dimension of the space such that $\left\|x\right\|_{p}\leq c_{p,q}\left\|x\right\|_{q}$ for all $x\in\mathbb{R}^{d\cdot k}$.

Theorem 12

Let $p\in[1;\infty]$, $\star\in\{\pm,=\}$ and $z^{(p)}\in\mathbb{R}^{d\cdot k}$ be an optimal solution of (3). For all $q\in[1;\infty]$ the vector $z^{\prime}=c_{p,q}z^{(p)}$ satisfies $\mathbb{B}^{q}_{1}(z^{\prime})\subseteq N_{\mathcal{P}^{\star}}(w(C))$. Moreover, its objective value satisfies $\left\|z^{\prime}\right\|^{2}_{2}\leq\max\{c_{p,q},c_{q,p}\}^{2}\left\|z^{(q)}\right\|^{2}_{2}$ where $z^{(q)}\in\mathbb{R}^{d\cdot k}$ is the optimal solution of (3) w.r.t. the $q$-norm.

Note that an upper bound on the objective value of the approximate solution $z^{\prime}$ implies a lower bound on its stability $\tau^{p}_{\star}(z^{\prime})$ and that $d(z^{\prime},z^{(p)})=0$, i.e. $z^{\prime}$ and $z^{(p)}$ define the same separating power diagram. If we choose $p=2$, then the auxiliary problems (4) of Theorem 11 have optimal objective values $\gamma_{j}=-\left\|v_{j}\right\|_{2}$ for all $j\in[t]$. Problem (3) then reduces to

[TABLE]

After computing an optimal solution $z^{(2)}$ of (5), we get a site vector $z^{\prime}=c_{2,p}z^{(2)}$ for any arbitrary $p\in[1;\infty]$ whose norm can be bounded from above by the norm of the most stable (w.r.t. $p$) site vector $z^{(p)}$ and a constant factor only depending on $p$ (Theorem 12). Hence, we obtain a provable approximation.

Proof-of-Concept Computations

We use Figures 12 – 17 to illustrate computational results for the the two clusterings of Figure 8. We compute the optimal sites (crosses in respective colors), following the programs in Theorem 11, w.r.t. $p=2,\infty$ for the all-shape and single-shape partition polytopes. The $6$-dimensional optimal solutions of (3) were again scaled to Euclidean norm 4. Figures 12 and 13 depict the area of possible perturbation without changing the clustering when only perturbing the first (black) site and keeping the other two sites fixed.

In Figures 14 – 17, all sites can be perturbed simultaneously within the drawn $p$-norm balls without changing the clustering. The different sizes of the $p$-norm balls are due to scaling of the optimal solutions of (3). Note that scaling the sites does not change the clustering, but of course, the scaling does affect the radius of the $p$-norm ball. In fact, the radius directly corresponds to the stability of the depicted optimal sites. The BHR stability of the left-hand clustering is $\tau^{2}_{\pm}(z^{(2)})\approx 0.253$ and $\tau^{\infty}_{\pm}(z^{(\infty)})\approx 0.139$, whereas for the right-hand clustering we get $\tau^{2}_{\pm}(z^{(2)})\approx 0.018$ and $\tau^{\infty}_{\pm}(z^{(\infty)})\approx 0.009$. Note that on the right-hand side of Figures 14 and 15, the balls around each cross that indicate the area of possible perturbation are so small that they become hard to see.

Figures 16 and 17 depict the areas of perturbation for the single-shape partition polytope. The difference between left-hand and right-hand clustering of the LSA stability is not as large as in the previous examples, since the cluster sizes are fixed. We obtain $\tau^{2}_{=}(z^{(2)})\approx 0.365$, $\tau^{\infty}_{=}(z^{(\infty)})\approx 0.163$ (left) and $\tau^{2}_{=}(z^{(2)})\approx 0.199$, $\tau^{\infty}_{=}(z^{(\infty)})\approx 0.100$ (right). Nevertheless, one sees that the sites on the left-hand side can be perturbed more than the ones on the right-hand side.

4 Proofs

In this section, we present proofs of theorems stated in Sections 3.1 – 3.3. We will prove equivalent statements for $\mathcal{P}^{\pm}$ and $\mathcal{P}^{=}$ just for the bounded-shape case, since they work analogously. It would suffice to replace “$\pm$” by “$=$”.

4.1 Proofs for Section 3.1

Proof 2

Proof of Theorem 7. A clustering $C$ is not a vertex clustering if and only if $\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))=\emptyset$, yielding vol$(N_{\mathcal{P}^{\pm}}(w(C)))=0$ and $\mu_{\pm}(C)=0$. So let $C$ be a vertex clustering.

If $a\in N_{\mathcal{P}^{\pm}}(w(C))$, then $\frac{1}{\left\|a\right\|_{2}}a\in N_{\mathcal{P}^{\pm}}(w(C))\cap\mathbb{S}^{d\cdot k}$. Suppose we are given a site vector $z\in\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))\cap\mathbb{S}^{d\cdot k}$. Then $z$ is contained in the relative interior of $B(N_{\mathcal{P}^{\pm}}(w(C)))$. The base of a cone is spherically convex, so for all $a\in B(N_{\mathcal{P}^{\pm}}(w(C)))$ the geodesic “connecting” $z$ and $\frac{1}{\left\|a\right\|_{2}}a$, i.e. the $\gamma$ satisfying $L(\gamma)=d(z,a)$, is contained in $B(N_{\mathcal{P}^{\pm}}(w(C)))$. The area of $B(N_{\mathcal{P}^{\pm}}(w(C)))$, which is equal to vol$(N_{\mathcal{P}^{\pm}}(w(C)))$, can be computed by an integral over all directions (geodesics through $z$) of $d(z,a)$ with $a\in B(N_{\mathcal{P}^{\pm}}(w(C)))$ maximizing $d(z,\cdot)$ along the direction of $\gamma$. In fact, starting at $z$, we can walk into any direction along the unit sphere for a small distance and still stay in the interior of the normal cone. In particular, for every $a\in\operatorname{bd}(B(N_{\mathcal{P}^{\pm}}(w(C))))\subseteq\operatorname{bd}(N_{\mathcal{P}^{\pm}}(w(C)))$, there is a geodesic $\gamma$ satisfying $L(\gamma)=d(z,a)>0$, and for every geodesic $\gamma$ starting at $z$ we have

[TABLE]

with $\{\bar{a}\}=\operatorname{bd}(B(N_{\mathcal{P}^{\pm}}(w(C))))\cap\gamma$. We obtain

[TABLE]

which, together with the fact that $\operatorname{vol}(\mathbb{R}^{d\cdot k})$ is equal to the area of the surface of $\mathbb{S}^{d\cdot k}$, yields representation (2).

Let $K\subseteq\mathbb{R}^{d\cdot k}$ be a cone with $\operatorname{vol}(K)=\operatorname{vol}(N_{\mathcal{P}^{\pm}}(w(C)))$ and consider a diffeomorphism $f:B(N_{\mathcal{P}^{\pm}}(w(C)))\mapsto B(K)$. Then $f(z)$ satisfies $d(f(z),a)>0$ for all $a\in\operatorname{bd}(B(K))$, as open sets (in $B(N_{\mathcal{P}^{\pm}}(w(C)))$) are mapped to open sets (in $B(K)$) and vice versa. Furthermore, as $\operatorname{vol}(K)=\operatorname{vol}(N_{\mathcal{P}^{\pm}}(w(C)))$ and the bases $B(K)$ and $B(N_{\mathcal{P}^{\pm}}(w(C)))$ are spherically convex, we can perturb $f(z)$ and $z$ in equal measure without leaving the bases of the respective cones. This shows that $\mu_{\pm}$ as a quality measure is well-defined. \qed

4.2 Proofs for Section 3.2

In this section, we work towards a proof of Theorem 8. First, we need a few auxiliary lemmas.

Fix $C,C^{\prime}$ to be the clusterings of Theorem 8. Let $CDG(C,C^{\prime})$ be their clustering difference graph and let $\mathcal{M}$ be the set of movements corresponding to a decomposition of $CDG(C,C^{\prime})$ into cycles and paths. Denote by $\mathcal{M}_{\geq 2}\subseteq\mathcal{M}$ the set of movements corresponding to cycles and non-trivial paths, i.e. paths (and cycles) containing at least two edges.

Lemma 13

Let $a\in\mathbb{R}^{d\cdot k}$ such that $a^{T}w(C)=a^{T}w(C^{\prime})>a^{T}w$ for all other vertices $w\in\mathcal{P}^{\pm}\setminus\{w(C),w(C^{\prime})\}$. Then $a^{T}w(M)=0$ for all $M\in\mathcal{M}$.

Proof 3

Proof. Let $M\in\mathcal{M}$ and suppose $a^{T}w(M)>0$. Applying $M$ to $C$ yields a (feasible) clustering $\bar{C}$ with $a^{T}w(\bar{C})=a^{T}(w(C)+w(M))>a^{T}w(C)$ which is a contradiction. For $a^{T}w(M)<0$ we obtain the same contradiction by applying $M^{-1}$ to $C^{\prime}$, as this would yield a (feasible) clustering $\bar{C}$ with

[TABLE]

So $a^{T}w(M)=0$. \qed

This implies that all movements in $\mathcal{M}$ operate on the same clusters.

Lemma 14

There exists an index subset $I:=\{i_{1},\dots,i_{t}\}\subseteq[k]$ such that all paths and cycles in $CDG(C,C^{\prime})$ corresponding to (cyclical) movements in $\mathcal{M}$ contain all nodes in $I$ and no other.

Proof 4

Proof. By Lemma 13, the vectors of all movements in $\mathcal{M}$ are collinear. Recall that $X$ consists of distinct, non-zero data points. So, if movements would move items between different subsets of clusters, their corresponding vectors would leave different components of the clustering vectors unchanged. But then the vectors of these movements would not be collinear.

Let $M_{1}:C_{i_{1}}\rightarrow C_{i_{2}}\rightarrow\cdots\rightarrow C_{i_{t}}$ be a movement in $\mathcal{M}$ for some $2\leq t\leq k$. Note that if $t\geq 3$ and $i_{1}=i_{t}$, then $M_{1}$ is a cyclical movement. Suppose there exists a movement $M_{2}$ whose corresponding path in the CDG contains an edge $(i_{l},i_{j})$ with $j\not=l+1$ (cyclical indexing). Then the paths in $CDG(C,C^{\prime})$ take one of the forms illustrated in Figure 18. Figure 18(a) shows the path corresponding to $M_{1}$.

For $l+1<j$, $M_{2}$ skips all clusters between $i_{l}$ and $i_{j}$, see Figure 18(b). For $l>j$ we obtain a cyclical movement skipping all clusters $C_{i_{r}}$ with $r<j$ or $r>l$, c.f. Figure 18(c). In both cases, there exists a (cyclical) movement that leaves different components of the clustering vectors unchanged. In particular, $M_{2}$ does not change cluster $i_{l+1}$, but $M_{1}$ does. Therefore, the vectors $w(M_{1})$ and $w(M_{2})$ cannot be collinear, contradicting Lemma 13. \qed

Lemma 15

If no three points in $X$ lie on a single line, then $\left|\mathcal{M}_{\geq 2}\right|\leq 1$.

Proof 5

Proof. Assume by contradiction that $\left|\mathcal{M}_{\geq 2}\right|\geq 2$, i.e. there are at least two non-trivial paths or cycles in $CDG(C,C^{\prime})$. All movements in $\mathcal{M}_{\geq 2}$ operate on the same clusters (Lemma 14) and their vectors are collinear (Lemma 13). Suppose there are two movements $M_{1},M_{2}\in\mathcal{M}_{\geq 2}$ of the form

[TABLE]

for some $3\leq t\leq k$ and $l\in[t]$ (cyclical indexing). Note that, if $i_{1}=i_{t}$ or $i_{l-1}=i_{l}$, then $M_{1}$ or $M_{2}$ corresponds to a cycle, respectively. In this case, the element $x^{\prime}_{t}$, respectively $x_{i_{l-1}}$, does not exist. Both paths corresponding to $M_{1}$ and $M_{2}$ have to contain the same nodes in the same order, otherwise we would obtain a contradiction as in the proof of Lemma 14.

Since every $x\in X$ is moved at most once, we know $x_{i_{r}}\not=x^{\prime}_{i_{j}}$ for all $r\in[t-1]$ and $j\in[t]\setminus\{l-1\}$. Consider the part of $CDG(C,C^{\prime})$ that contains both paths, c.f. Figure 19(a). We can construct two movements $M^{\prime}_{1}$ and $M^{\prime}_{2}$, as explained in Figure 19.

In Figure 19(b), we see that $M^{\prime}_{j}$ equals $M_{j}$ for $j=1,2$ between the nodes $i_{t}$ and $i_{l}$. Between the $i_{l}$-th and the $i_{t}$-th cluster, $M^{\prime}_{1}$ equals $M_{2}$ and $M^{\prime}_{2}$ equals $M_{1}$. The clustering we obtain after applying the movements $M^{\prime}_{1}$ and $M^{\prime}_{2}$ to $C$ equals the one after applying $M_{1}$ and $M_{2}$ to $C$. Thus we could replace the paths corresponding to $M_{1}$ and $M_{2}$ by the ones corresponding to $M^{\prime}_{1}$ and $M^{\prime}_{2}$ in the decomposition of $CDG(C,C^{\prime})$. By Lemma 13, the vectors $w(M_{1})$, $w(M_{2})$, $w(M^{\prime}_{1})$ and $w(M^{\prime}_{2})$ are collinear. Let us now consider the components of $w(M_{1})$ and $w(M^{\prime}_{1})$ corresponding to the changes of $\sigma_{i_{l}}$. We obtain for $w(M_{1})$ and $w(M^{\prime}_{1})$ (in this order)

[TABLE]

which are collinear. But this implies that $x_{i_{l-1}},x_{i_{l}}$ and $x^{\prime}_{i_{l}}$ lie on a single line, a contradiction. Hence, $\left|\mathcal{M}_{\geq 2}\right|<2$. \qed

In particular, $CDG(C,C^{\prime})$ contains at most one cycle or path of edge-length greater than 1.

Proof 6

Proof of Theorem 8. If $\left|C\right|=\left|C^{\prime}\right|$, then the statement follows from Lemma 15 and the fact that $C\not=C^{\prime}$ implies that $CDG(C,C^{\prime})$ contains at least one cycle. So suppose $\left|C\right|\not=\left|C^{\prime}\right|$. If $CDG(C,C^{\prime})$ contains a path $(i_{1},i_{2})-(i_{2},i_{3})-\cdots-(i_{t-1},i_{t})$ with $t\geq 3$, then, by Lemma 15, there does not exist any other paths or cycle. Now let $t=2$ and suppose there are three movements $M_{1},M_{2}$ and $M_{3}$, which all are of the form $C_{i_{1}}\rightarrow C_{i_{2}}$. By collinearity of the vectors $w(M_{1})$, $w(M_{2})$ and $w(M_{3})$, the respective components corresponding to $\sigma_{i_{2}}$ are collinear. By definition, these entries equal the labels of the three edges in $CDG(C,C^{\prime})$ – which are collinear. This is a contradiction to the assumption that no three points in $X$ lie on a single line. \qed

Proof 7

Proof of Corollary 9. Note that $\left|C\right|=\left|C^{\prime}\right|$ implies that $\mathcal{M}=\mathcal{M}_{\geq 2}$, since the CDG decomposes into cycles. Suppose there exist two cycles in $CDG(C,C^{\prime})$ with corresponding cyclical movements $M_{1}$ and $M_{2}$ of the form

[TABLE]

for some $t\geq 2$. Again, we have $x_{j_{l}}\not=x^{\prime}_{j_{r}}$ for all $l,r\in[t]$. Analogously to the proof of Lemma 15, we can construct two different cycles $M^{\prime}_{1}$ and $M^{\prime}_{2}$ via $M^{\prime}_{j}=M_{j}$ between $i_{1}$ and $i_{t}$ and $M^{\prime}_{1}=M_{2}$ and $M^{\prime}_{2}=M_{1}$ between clusters $C_{i_{t}}$ and $C_{i_{1}}$. The corresponding vectors $w(M_{1})$, $w(M_{2})$, $w(M^{\prime}_{1})$ and $w(M^{\prime}_{2})$ are collinear and, considering the $d$ entries corresponding to $\sigma_{i_{1}}$, we obtain (in this order)

[TABLE]

which are collinear. This implies that $x_{j_{1}}$, $x^{\prime}_{j_{1}}$, $x_{j_{t}}$ and $x^{\prime}_{j_{t}}$ lie on a line, a contradiction. \qed

4.3 Proofs for Section 3.3

Proof 8

Proof of Theorem 11. Let $p\in[1;\infty]$, let $C$ be a vertex clustering and let $N_{\mathcal{P}^{\pm}}(w(C))$ be its normal cone with facets $F_{1},\dots,F_{t}$. The “gravity center” at $0\in\mathbb{R}^{d\cdot k}$ which attracts the $p$-norm ball can be modeled by minimizing the squared Euclidean norm $\left\|z\right\|_{2}^{2}$. Using $dist(z,F_{j})_{p}=\inf\{\left\|z-f^{j}\right\|_{p}\ |\ f^{j}\in F_{j}\}$ to denote the distance of $z$ and the facet $F_{j}$ w.r.t. the $p$-norm, constraints $dist(z,F_{j})_{p}\geq 1$ for all $j\in[t]$ guarantee that a $p$-norm unit ball with center $z$ remains inside the normal cone $N_{\mathcal{P}^{\pm}}(w(C))$. This gives the optimization problem

[TABLE]

First we show that an optimal solution $z^{(p)}$ of (6) has the highest stability w.r.t. $p$ among all vectors in $N_{\mathcal{P}^{\pm}}(w(C))$.

The stability of $z^{(p)}$ is equal to $\tau^{p}_{\pm}(z^{(p)})=\frac{1}{\left\|z^{(p)}\right\|_{2}}>0$ by construction. Note that any site vector on the boundary of the normal cone has stability [math]. So let $z\in\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))$ and let $\delta_{z}:=\min\{dist(z,F_{j})_{p}\ |\ j\in[t]\}>0$ be the smallest $p$-norm distance to any of the facets. Then $\tau^{p}_{\pm}(z)=\frac{\delta_{z}}{\left\|z\right\|_{2}}$. Moreover, the vector $\frac{1}{\delta_{z}}z$ is feasible for (6) and, by optimality of $z^{(p)}$, we obtain

[TABLE]

Next, we prove that (6) is equivalent to (3). First we motivate the choices of $\gamma_{1}^{p},\dots,\gamma_{t}^{p}$. The “distance constraints” of (6) define two half spaces for each $j\in[t]$. Formally, we would have to replace $F_{j}$ by $\operatorname{lin}(F_{j})$, but this becomes irrelevant due to only considering vectors in the normal cone. With $z^{*}_{j}\in B^{p}_{1}(0)$ being an optimal solution and $\gamma_{j}^{p}:=v_{j}^{T}z^{*}_{j}<0$ its objective value of (4), we obtain the equivalence

[TABLE]

Furthermore, for all $j\in[t]$ we know that $F_{j}=\operatorname{lin}(F_{j})\cap N_{\mathcal{P}^{\pm}}(w(C))$, where $\operatorname{lin}(F_{j})=\{v_{j}\}^{\perp}$ is a $(d\cdot k-1)$-dimensional linear subspace, i.e. a hyperplane through the origin with normal vector $v_{j}$. In particular, we can write any $z\in\mathbb{R}^{d\cdot k}$ as linear combination of an orthogonal basis $\{v_{j},f^{j}_{1},\dots,f^{j}_{d\cdot k-1}\}$ of $\mathbb{R}^{d\cdot k}$ with $f^{j}_{l}\in\operatorname{lin}(F_{j})$ for all $l\in[d\cdot k-1]$, i.e.

[TABLE]

For $z\in N_{\mathcal{P}^{\pm}}(w(C))$, we observe that $\nu^{j}_{0}\leq 0$, since $v_{j}$ points along an edge direction, and thus away from the normal cone. For $z\in\operatorname{int}(N_{\mathcal{P}^{\pm}}(w(C)))$, we observe that $\nu_{0}^{j}<0$. Together with (7), we obtain

[TABLE]

for all $j\in[t]$. Rewriting the normal cone as

[TABLE]

and recalling that $\gamma_{j}^{p}<0$, we see that the halfspace $H^{\leq}_{(v_{j},\gamma_{j}^{p})}$ is strictly contained in $H^{\leq}_{(v_{j},0)}$ for all $j\in[t]$. Thus,

[TABLE]

Therefore, the constraint $z\in N_{\mathcal{P}^{\pm}}(w(C))$ is redundant and (6) is equivalent to (3). \qed

We choose the $p$-norm distance to the facets of the cone as a constraint in (6), because lower dimensional faces blocking the ball do not guarantee a stable site vector. To see this, consider the following $3$-dimensional example for $p=2$ and suppose the edges of the cone are used to block the unit ball rather than the facets.

Let $z=(0,0,2)^{T}$, $a^{1}=(0,\sqrt{3},3)^{T}$, $a^{2}=(0,-\sqrt{3},3)^{T}$, $a^{3}=(\sqrt{3},0,3)^{T}\in\mathbb{R}^{3}$ and consider the cone spanned by $\{a^{1},a^{2},a^{3}\}$, see Figure 20 for cross-sections orthogonal to the $x_{1}$-axis (left) and orthogonal to the $x_{3}$-axis (right). Note that $a^{1},a^{2}$ and $a^{3}$ are linearly independent. We have $z=\frac{1}{3}\cdot a^{1}+\frac{1}{3}\cdot a^{2}+0\cdot a^{3}$, which implies $z\in\operatorname{bd}(\operatorname{pos}(\{a^{1},a^{2},a^{3}\})$. One can easily verify that $dist(z,\operatorname{pos}(\{a^{j}\}))_{2}=\left\|z-\frac{1}{2}a^{j}\right\|_{2}=1$ for all $j\in[3]$, i.e. $z$ is feasible and all constraints are tight, if we consider the edges instead of the facets of the cone in (6). Note that we cannot decrease the (squared) norm of $z$ any further. However, this optimal solution has stability $\tau^{2}_{\star}(z)=0$, i.e. this site vector cannot be perturbed in every direction without leaving the cone.

Similar examples can be constructed for any $p\in[1;\infty]$ and any $l$-dimensional face of $N_{\mathcal{P}^{\pm}}(w(C))$ that is not a facet. The case $l=0$ (the origin blocking the ball) does not provide any information, because every center has $p$-norm distance one to the origin. Let us consider the difference between using $l$-dimensional faces and using the facets $F_{1},\dots,F_{t}$ of the normal cone to block the unit ball:

For every $j\in[t]$, the linear subspace $\operatorname{lin}(F_{j})$ is a hyperplane that separates the underlying space into two halfspaces. In contrast, the linear subspace $\operatorname{lin}(F)$ of an $l$-dimensional face $F$ ($1\leq l<d\cdot k-1$) is not a hyperplane. This implies that a representation of the “distance constraints” $dist(z,\operatorname{lin}(F))_{p}\geq 1$ is not possible when using lower-dimensional faces. Therefore, the arguments of the proof of Theorem 11 do not apply to lower dimensional faces. Informally, the center of the ball can “orbit” $\operatorname{lin}(F)$ at $p$-norm distance $1$ and eventually “hit” the boundary of the cone.

Proof 9

Proof of Theorem 12. Let $q\in[1;\infty]$ with $q\not=p$, else the claim is trivial, because $c_{p,p}=1$. By assumption $\mathbb{B}^{p}_{1}(z^{(p)})\subseteq N_{\mathcal{P}^{\pm}}(w(C))$. Thus $\mathbb{B}^{p}_{\lambda}(\lambda z^{(p)})\subseteq N_{\mathcal{P}^{\pm}}(w(C))$ for all $\lambda>0$. It suffices to show that $\lambda=c_{p,q}$ satisfies $\mathbb{B}^{q}_{1}(0)\subseteq\mathbb{B}^{p}_{c_{p,q}}(0)$. For $y\in\mathbb{B}^{q}_{1}(0)$ we obtain

[TABLE]

Hence, choosing $\lambda=c_{p,q}>0$ yields $\mathbb{B}^{q}_{1}(c_{p,q}z^{(p)})\subseteq\mathbb{B}^{p}_{c_{p,q}}(c_{p,q}z^{(p)})\subseteq N_{\mathcal{P}^{\pm}}(w(C))$.

Now let $z^{(q)}$ be the optimal solution of (3) w.r.t. the $q$-norm, i.e. with constraints $dist(z,F_{j})_{q}\geq 1$ for all $j\in[t]$. In the following, whenever we refer to (3), we refer to the optimization problem w.r.t. the $p$-norm. Let $\gamma^{q}_{1},\dots,\gamma^{q}_{t}$ denote the optimal objective values of (4) with $\mathbb{B}^{q}_{1}(0)$ as feasible region, and let $\gamma^{p}_{1},\dots,\gamma^{p}_{t}$ denote the same for $\mathbb{B}^{p}_{1}(0)$. We distinguish between the two cases $p<q$ and $p>q$.

For $p<q$, we know $\mathbb{B}_{1}^{p}(0)\subseteq\mathbb{B}^{q}_{1}(0)$ and, therefore, $z^{(q)}$ satisfies $v_{j}^{T}z^{(q)}\leq\gamma^{q}_{j}\leq\gamma^{p}_{j}$ for all $j\in[t]$. Hence, $z^{(q)}$ is feasible for (3) and $\mathbb{B}^{p}_{1}(z^{(q)})\subseteq N_{\mathcal{P}^{\pm}}(w(C))$. The objective value of $z^{\prime}=c_{p,q}z^{(p)}$ satisfies

[TABLE]

because $z^{(p)}$ is an optimal solution of (3).

For $p>q$, consider $\bar{z}=c_{q,p}z^{(q)}$, which satisfies $\mathbb{B}^{p}_{1}(\bar{z})\subseteq N_{\mathcal{P}^{\pm}}(w(C))$. Note that $z^{(q)}$ is not feasible for (3), but $\bar{z}$ is. Since $p>q$, we have $c_{p,q}=1$, i.e. $z^{\prime}=z^{(p)}$ and hence, by optimality of $z^{(p)}$,

[TABLE]

In total, since either $c_{p,q}$ or $c_{p,q}$ equals 1, (8) and (9) yield

[TABLE]

This proves the claim. \qed

5 Concluding Remarks

We would like to stress that all computations of the volume of the normal cone, the computations of edges of the normal cone, as well as the optimal solution of (3) are challenging due to the possibly exponential number of edges incident to a vertex of the polytope. However, this significant computational effort is rewarded with deep insights into the structure and behavior of site vectors. The key takeaway is that clusterings of large volume, i.e. vertex clustering with large normal cones, are both good and likely to be found by clustering algorithms. The next natural steps include the design of approximation algorithms that trade small errors in the construction of normal cones for a significant reduction in computation times.