Assessing One-Dimensional Cluster Stability by Extreme-Point Trimming

TL;DR

This paper introduces a probabilistic method for evaluating the stability of one-dimensional data clusters by analyzing how their range contracts when extreme points are trimmed, providing theoretical insights and practical tools.

Contribution

It develops a new shrinkage-based test for cluster stability, with analytical expressions and a clustering validation pipeline, outperforming classical methods in small-sample and noisy conditions.

Findings

The method accurately distinguishes distribution types using shrinkage curves.

The proposed test outperforms likelihood-ratio tests in small samples.

It effectively validates one-dimensional clusters within a clustering pipeline.

Abstract

We develop a probabilistic method for assessing the tail behavior and geometric stability of one-dimensional n i.i.d. samples by tracking how their span contracts when the most extreme points are trimmed. Central to our approach is the diameter-shrinkage ratio, that quantifies the relative reduction in data range as extreme points are successively removed. We derive analytical expressions, including finite-sample corrections, for the expected shrinkage under both the uniform and Gaussian hypotheses, and establish that these curves remain distinct even for moderate number of removal. We construct an elementary decision rule that assigns a sample to whichever theoretical shrinkage profile it most closely follows. This test achieves higher classification accuracy than the classical likelihood-ratio test in small-sample or noisy regimes, while preserving asymptotic consistency for large n. We further integrate our criterion into a clustering pipeline (e.g. DBSCAN), demonstrating its ability to validate one-dimensional clusters without any density estimation or parameter tuning. This work thus provides both theoretical insight and practical tools for robust distributional inference and cluster stability analysis.

Figures

Tables

Table 1. Table 1 : Comparison of 2 2 different versions of the approximation of the expected value of the position of the k t ​ h k^{th} order position of n n Gaussian i.i.d., for different values of k k close to n n .

$k$	n	$n-1$	$n-2$	$n-3$	$n-4$
$u_k$	$3.12389802$	$2.87248195$	$2.74677391$	$2.66296856$	$2.60011454$
$v_k$	$3.12389802$	$2.87220937$	$2.73608587$	$2.64103412$	$2.56728926$

Table 2. Table 2 : Aggregate performance of the MLE versus shrinkage‐based classifiers across varying sample sizes.

Method	Mean Accuracy	AUC (ROC)
MLE (likelihood‐ratio)	0.851	0.851
Shrinkage ($T_{\mathrm{shrink}}^{\left(i\le p\right)},p=6$)	0.864	0.864

Equations

$$D_p:=X_{(n-p)}-X_{(p+1)},$$

$$T_{\text{shrink}}^{(p)}:=\frac{D_p}{D_{p-1}},$$

$$\mathbb{E}(T_{\text{shrink}}^{(p)})\approx \frac{\mathbb{E}(D_p)}{\mathbb{E}(D_{p-1})},$$

$$f(x_1,\dots ,x_n)=\{\begin{array}{ll}\frac{n!}{L^n}& \text{if }0\le x_1\le \cdots \le x_n\le L,\\ 0& \text{otherwise}.\end{array}$$

$$f_k(x_k)$$

$$=\frac{n!}{L^n}\left({\int }_0^{x_k}d{x}_1{\int }_{x_1}^{x_k}d{x}_2\cdots {\int }_{x_{k-2}}^{x_k}d{x}_{k-1}\right)\left({\int }_{x_k}^L{d}_{x_{k+1}}{\int }_{x_{k+1}}^L{d}_{x_{k+2}}\cdots {\int }_{x_{n-1}}^{L}d{x}_n\right)$$

$$=\frac{n!}{L^n}L(x_k)R(x_k),$$

$$L(x_k)={\int }_0^{x_k}d{x}_1{\int }_{x_1}^{x_k}d{x}_2\cdots {\int }_{x_{k-2}}^{x_k}d{x}_{k-1}=\frac{x_k^{k-1}}{(k-1)!}$$

$$R(x_k)={\int }_{x_k}^L{d}_{x_{k+1}}{\int }_{x_{k+1}}^L{d}_{x_{k+2}}\cdots {\int }_{x_{n-1}}^{L}d{x}_n=\frac{(L-x_k{)}^{n-k}}{(n-k)!}.$$

$$f_k(x)=\frac{n!}{(k-1)!(n-k)!}\cdot \frac{x^{k-1}(L-x{)}^{n-k}}{L^n},x\in [0,L].$$

$$\mathbb{E}(X_{(k)})=\frac{k}{n+1}\cdot L$$

$$\mathbb{E}(D_p)=\mathbb{E}(X_{(n-p)})-\mathbb{E}(X_{(p+1)})=\frac{n-2p-1}{n+1}\cdot L.$$

$$\mathbb{E}(T_{\text{shrink}}^{(p)})\approx \frac{n-2p-1}{n-2p+1}.$$

$$\bar{X}=\frac{1}{n}\sum _{i=1}^n{X}_{(i)},\mathbb{E}[\bar{X}]=\frac{1}{n}\sum _{i=1}^n\mathbb{E}[X_{(i)}]=\frac{n(n+1)L}{2n(n+1)}=\frac{L}{2}$$

$$\mathbb{E}[{\bar{X}}_{-}]=\frac{1}{n-1}\sum _{i=1}^{n-1}\mathbb{E}[X_{(i)}]=\frac{n(n-1)}{2(n-1)(n+1)}=\frac{nL}{2(n+1)}.$$

$$\mathbb{E}[\bar{X}-{\bar{X}}_{-}]=\frac{L}{2}-\frac{nL}{2(n+1)}=\frac{L}{2(n+1)}.$$

$$\mathbb{E}\bigl{[}\bar{X}-\bar{X}_{-}\bigr{]}=\frac{L}{2(n+1)}.$$

$$|S_n^{\max }|=\underset{1\le k\le n}{\max }|S_k|.$$

$$\displaystyle\mathbb{E}\bigl{[}\,|S_{n}^{\max}|\bigr{]}\;\sim\ \sigma\sqrt{\tfrac{\pi\,\ln n}{2}}.$$

$$\Pr\{|S|>R\}=1-\Pr\{|S|\leq R\}=1-\mbox{\boldmath$erf$}\!\Bigl{(}\tfrac{R}{\sigma\sqrt{2}}\Bigr{)},$$

$$\Pr\{|S_{n}^{\max}|>R\}=1-\Pr\{|S|\leq R\}^{n}=1-\mbox{\boldmath$erf$}\!\Bigl{(}\tfrac{R}{\sigma\sqrt{2}}\Bigr{)}^{n}.$$

$$\mathbb{E}\bigl{[}|S_{n}^{\max}|\bigr{]}=\int_{0}^{\infty}\Pr\{|S_{n}^{\max}|>r\}\,dr=\int_{0}^{\infty}\Bigl{[}\,1-\mbox{\boldmath$erf$}\!\bigl{(}\tfrac{r}{\sigma\sqrt{2}}\bigr{)}^{n}\Bigr{]}\,dr.$$

$$\mbox{\boldmath$erf$}\!\bigl{(}\tfrac{r}{\sigma\sqrt{2}}\bigr{)}^{n}\approx\Bigl{(}1-e^{-\,\tfrac{2r^{2}}{\pi\sigma^{2}}}\Bigr{)}^{\!n/2}.$$

$$I_{n}=\int_{0}^{\infty}\Bigl{[}\,1-\bigl{(}1-e^{-2r^{2}/(\pi\sigma^{2})}\bigr{)}^{n/2}\Bigr{]}\,dr,$$

$$u=\sqrt{\frac{2}{\pi {\sigma }^2}}r,dr=\sigma \sqrt{\frac{\pi }{2}}du.$$

$$I_{n}=\sigma\sqrt{\tfrac{\pi}{2}}\int_{0}^{\infty}\Bigl{[}\,1-(1-e^{-u^{2}})^{n/2}\Bigr{]}\,du.$$

$$I_{n}=\sigma\sqrt{\tfrac{\pi\,\ln n}{2}}\int_{0}^{\infty}\Bigl{[}\,1-\bigl{(}1-n^{-v^{2}}\bigr{)}^{n/2}\Bigr{]}\,dv.$$

$$\bigl{(}1-n^{-v^{2}}\bigr{)}^{n/2}=\exp\bigl{(}\tfrac{n}{2}\ln(1-n^{-v^{2}})\bigr{)}\;\longrightarrow\;\begin{cases}0,&0\leq v<1,\\ 1,&v>1,\end{cases}$$

$$\int_{0}^{\infty}\Bigl{[}1-(1-n^{-v^{2}})^{n/2}\Bigr{]}\,dv\;\longrightarrow\;1.$$

$$\mathbb{E}[|S_n^{\max }|]\sim I_n\sim \sigma \sqrt{\frac{\pi \ln n}{2}}.$$

$$\mathbb{E}[|S_n^{\max }|]\sim \mathbb{E}[X_n]\sim \sigma \sqrt{2\log n},$$

$$f(x_1,\dots ,x_n)=\{\begin{array}{ll}\frac{n!}{(\sigma \sqrt{2\pi }{)}^n}\exp \left(-\frac{1}{2{\sigma }^2}{\sum }_{i=1}^n{x}_i^2\right)& \text{if }{x}_1\le \cdots \le x_n,\\ 0& \text{otherwise}.\end{array}$$

Full text

1 Introduction

The automated identification of clusters or isolated points is a fundamental step in many classification and spatial analysis pipelines [1, 2, 3] to identify structures in unlabeled data. Clustering typically begins by assigning labels to data points, indicating their membership to one or more groups. However, the strategies used to define these groups can vary significantly across clustering methods, depending on the underlying assumptions about data structure, density, or similarity.

Clustering and classification algorithms can be broadly categorized into partitioning-based, hierarchical, and density-based methods. Partitioning methods, such as K-means [4, 5], Spectral Clustering [6], and Support Vector Machines (SVMs) [7], divide the data into distinct groups by optimizing specific criteria. K-means partitions data into a fixed number of spherical clusters by minimizing within-cluster variance. Spectral Clustering extends partitioning by leveraging the eigenstructure of similarity graphs to identify clusters with complex, non-convex shapes through an embedding step followed by a partitioning algorithm. Similarly, SVMs perform classification by implicitly mapping data into higher-dimensional feature spaces using the kernel trick, effectively partitioning data through linear separation in that transformed space. Unlike partitioning methods, hierarchical clustering builds a nested structure of clusters either from the bottom up (agglomerative, like Ward’s method [8] or from the top down (divisive [9]). In contrast, density-based methods, such as DBSCAN [10], identify clusters as dense regions of points separated by areas of lower density, allowing the discovery of arbitrarily shaped clusters and handling noise naturally, without needing to predefine the number of clusters.

Together, these techniques offer complementary strengths, depending on the shape, scale, and noise level of the data. In applications where point clouds represent high-dimensional embeddings—such as cell representations extracted from deep learning models—unsupervised clustering plays a crucial role in uncovering latent structure prior to any downstream statistical inference. Identifying clusters automatically in such settings enables spatial statistical analyses to infer local organizational principles, as in [3], where clusters correspond to cellular subtypes or functional microenvironments used to study how subtypes colonize brain area. Identifying clustering across a large population is thus a key step to generate the statitics.

A persistent challenge, however, is the lack of an absolute definition of what constitutes a “cluster.” Indeed, in many spatial statistics frameworks, a cluster is meaningful only in contrast to a reference null model, often based on homogeneous Poisson or binomial point processes [11].

Two primary approaches have been widely employed to determine whether a given cluster represents a true hotspot or merely a spurious pattern. The frequentist approach typically relies on hypothesis testing, often implemented via Monte Carlo simulations, to assess whether the null hypothesis can be rejected [12]. In contrast, the Bayesian approach evaluates the posterior probability of the null hypothesis relative to alternative hypotheses [13]. For both paradigms, significant efforts have been made to reduce computational costs, leading to various optimization strategies and algorithmic improvements [14, 15, 16].

Recent works in computational geometry suggest alternative, model-free methods to assess the stability of point ensembles. For instance, the evolution of convex hulls [17, 18], Voronoi cells [19], or nearest-neighbor graphs [20] can provide robust geometric indicators of underlying structure.

We focus here on examining the stability of an ensemble under the removal of extreme points—i.e., those furthest from the center of mass. This perspective has also emerged in clustering robustness studies and outlier detection literature [21, 22]. The present manuscrit is focusing in one dimension, ans we shall focus on how a segment length (diameter) and center of mass of a point cloud evolve as extreme points are removed successively. Intuitively, point clouds drawn from long-tailed distributions (e.g., Gaussian) are expected to undergo sharper geometric changes compared to those from compact distributions (e.g., uniform). We study here the evolution of geometric quantities for i.i.d. points in one dimension drawn from either a uniform distribution $\mathcal{U}[0,L]$ or a Gaussian distribution $\mathcal{N}(0,\sigma^{2})$. Specifically, in section 2, we derive the distribution and moments of order statistics for both the uniform and Gaussian cases. We particularly examine the expected shift in the center of mass and the behavior of the segment length (diameter) as extreme points are successively removed. In subsection 3.1, we introduce a family of geometric test statistics designed to discriminate between uniform and Gaussian samples. In particular, we propose an algorithm to assess the geometric stability of empirical one-dimensional distributions, which we benchmark against the likelihood ratio test. Finally, in subsection 3.2, we demonstrate how this algorithm can be applied to clusters identified by a spatial clustering algorithm.

These statistical test can be used for cluster validation, distributional testing, or as preprocessing steps for noise filtering and robust clustering. We show analytically and numerically that Gaussian-distributed ensembles are more sensitive to the removal of extreme points, revealing an intrinsic geometric fragility compared to uniformly distributed samples. This work bridges computational geometry, statistical inference, and robust clustering, and contributes to a growing literature on model-free geometric descriptors for point cloud analysis [23, 24, 25].

2 Order statistics, center of mass, and segment stability under point removal

In many statistical applications, particularly in unsupervised classification and geometric inference, a recurring question is whether an observed ensemble of points exhibits structural stability or reflects the influence of heavy-tailed variability. Motivated by this, we propose a method to assess the stability of point clouds in one dimension by iteratively removing extremal points—those furthest from the ensemble’s center of mass—and tracking the resulting geometric changes.

We start with $\mathcal{S}_{n}=\{x_{1},\dots,x_{n}\}\subset\mathbb{R}$ which is a sample of $n$ i.i.d. real-valued random variables. To formulate a statistical criterion for assessing the underlying distribution of such a sample, we introduce a procedure called the Diameter Shrinkage Statistic, which quantifies how the total length (i.e., the diameter) of the ensemble evolves as we remove the most extreme values.

We begin by defining the ordered sample $\mathcal{O}_{n}=\{X_{(1)}\leq\dots\leq X_{(n)}\}$, and for any integer $0\leq p<n/2$, we consider the segment length:

[TABLE]

corresponding to the length of the central segment obtained after removing the $p$ smallest and $p$ largest points from the sample. We define the diameter shrinkage ratio at step $p$ as:

[TABLE]

which reflects the relative contraction of the ensemble’s diameter due to the exclusion of the next outermost pair of points. Our goal is to characterize the distribution and expected value of this statistic under two reference models: the uniform distribution $\mathcal{U}[a,b]$ and the normal distribution $\mathcal{N}(\mu,\sigma^{2})$.

In the scope of this work, to initiate the study of the statistics $T_{\text{shrink}}^{(p)}$, rather than computing the exact expected value, we approximate the expected shrinkage ratio via the ratio of expected lengths (somehow justified by (93) in Appendix-A:

[TABLE]

thereby reducing the analysis to that of the order statistics $X_{(k)}$ for $1\leq k\leq n$. This approximation can be further examined in future work. In the next subsection, we compute formula 2 in the case of a Uniform Distribution statistics.

2.1 Shrinkage length ratio for a uniform Distribution

We first consider the case where the sample is drawn from the uniform distribution on the interval $[0,L]$. Rigorous computations extreme statistics are available [26, 27, 28] and we recall here elementary derivation to make this manuscript self contained. We recall that the joint probability density function of the ordered sample $X_{(1)},\dots,X_{(n)}$ is given by:

[TABLE]

The marginal density of the $k$-th order statistic $X_{(k)}$ can be computed by integrating this density over all the other variables under the ordering constraints:

[TABLE]

where the terms $L(x_{k})$ and $R(x_{k})$ respectively encode the cumulative integration over the $k-1$ variables smaller than $x_{k}$, and the $n-k$ variables greater than $x_{k}$. These terms can be computed by successive integrations of a polynomial function:

[TABLE]

The marginal density of the $k$-th order statistic $X_{(k)}$ is given by:

[TABLE]

The density function in Equation (9) is a scaled Beta distribution on interval $[0,L]$. Since the Beta $\displaystyle B$ and Gamma functions are connected by $\displaystyle B(\alpha,\beta)=\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$, the expected value of the $X_{(k)}$ variable is given by:

[TABLE]

The details of the computations are presented in Appendix-A see also [26, 27, 28]. Using this expression, the expected segment length shortening after removal of the $p$ smallest and $p$ largest points becomes:

[TABLE]

Consequently, the expected shrinkage ratio satisfies:

[TABLE]

Expression 11 is valid regardless of the bounds of the uniform distribution $\mathcal{U}[a,b]$. This provides a baseline expectation under the uniform model. Deviations from this behavior-—particularly under heavy-tailed distributions such as the Gaussian—-can then be used to construct hypothesis testing or stability metrics. In the following sections, we extend this analysis to Gaussian samples and demonstrate how the shrinkage statistic behaves differently, thus enabling the design of robust geometric tests for distributional classification.

2.2 Shift of the Sample Mean when Removing an Extreme Uniform Outlier

We now quantify how the sample mean (center of mass) is shifted when we discard the single largest observation from $n$ i.i.d. draws $X_{(1)}\leq\cdots\leq X_{(n)}\sim\mathcal{U}[0,L]$. By definition,

[TABLE]

Trimmed mean after removing the maximum.

Delete $X_{(n)}$ and form the $(n-1)$-point mean $\displaystyle\bar{X}_{-}=\frac{1}{n-1}\sum_{i=1}^{n-1}X_{(i)}.$ Its expectation is

[TABLE]

Therefore

[TABLE]

Net shift.

The expected shift in the center of mass upon removal of the single largest point is thus

[TABLE]

This expression stays valid regardless of the bounds of the uniform distribution $\mathcal{U}[a,b]$ by switching $L$ by $b-a$. By an identical argument for the smallest point—and by symmetry—the simultaneous removal of both extremes produces no first‐order shift in the mean.

2.3 Shrinkage length ratio for Gaussian Distribution

We now repeat the approach developed above to the case of a sampling $\mathcal{S}_{n}=\{x_{1},\dots,x_{n}\}\subset\mathbb{R}$ drawn for i.i.d. from a normal distribution $\mathcal{N}(0,\sigma^{2})$. In contrast to the uniform case, the analysis of order statistics for Gaussian variables is considerably more involved, as the literature providing closed-forme expressions for such order statistics distributions [29] often relies on general and technically sophisticated theorems like the Fisher-Tippett-Gnedenko theorem. Here, we aim to recover similar results using more elementary and self-contained computations, in the hope of providing a more accessible perspective. By combining integral decompositions with properties of the Gaussian cumulative and density functions, we will see below that we can derive close expressions.

2.3.1 Approximated Distribution of $|S^{max}_{n}|$

We present here an elementary asymptotic expression for the mean distance of the extreme point among n i.i.d Gaussian variable in ℝ. Thus $S_{1},\dots,S_{n}$ be i.i.d. $\mathcal{N}(0,\sigma^{2})$, and denote

[TABLE]

Our aim is to show from elementary computations that

[TABLE]

We start with the identify for $R\geq 0$,

[TABLE]

where \displaystyle\mbox{\boldmatherf}(x)=\frac{2}{\sqrt{\pi}}\int_{0}^{x}e^{-t^{2}}dt. Since $\{|S_{k}|\leq R\}$ are independent events,

[TABLE]

In addition,

[TABLE]

Using the approximation of $erf$ [30] (see Appendix-B for different asymptotic computation) \mbox{\boldmatherf}(x)\approx\sqrt{1-e^{-4x^{2}/\pi}},x\geq 0. Thus

[TABLE]

Set

[TABLE]

so $\mathbb{E}[|S_{n}^{\max}|]\approx I_{n}$. We now compute the integral by a first change of variables.

[TABLE]

Then

[TABLE]

followed by a second change of variables $u=v\sqrt{\ln n}$, so $du=\sqrt{\ln n}\,dv$. Hence

[TABLE]

Finally, we obtain the asymptotic via dominated convergence: for each fixed $v$,

[TABLE]

as $n\to\infty$. Since the integrand is bounded by 1, dominated convergence yields

[TABLE]

Therefore

[TABLE]

Although, in the literature, the large n limit gives

[TABLE]

[29] and Appendix-B. However, (14) is the asymptotic expression, for relatively small values of $n$ ($n\leq 10,000$), and the present formula (13) is closest to the simulations for n not that large. We therefore will use (13) in the following of this work.

2.3.2 Probability density function function of the $k$-th order statistic

We shall here derive an analytic expression for the probability density function function of the $k$-th order statistic. We start with the joint density of the ordered statistics $X_{(1)}\leq\dots\leq X_{(n)}$ for the i.i,d Gaussian variables. It is given by:

[TABLE]

The marginal density $f_{k}(x_{k})$ of the $k$-th order statistic, is obtained by integrating out all other variables under the ordering constraints:

[TABLE]

where

[TABLE]

Both $L(x_{k})$ and $R(x_{k})$ respectively accounts for the cumulative integration over the $k-1$ variables smaller than $x_{k}$, and the $n-k$ variables greater than $x_{k}$. We compute now $R(x_{k})$ by induction. Indeed,

[TABLE]

and

[TABLE]

Thus, through recursive integration, we obtain the general expression

[TABLE]

The term $L(x_{k})$ can be computed similarly using the same recursive integration:

[TABLE]

Finally, by combining these explicit formula 24-23, we obtain an explicit expression for the marginal density:

[TABLE]

2.3.3 Explicit expression for k-th order statistic ${E}[X_{(k)}]$

To compute the expectation of the $k$th order statistic $X_{(k)}$ of $n$ i.i.d. $\mathcal{N}(0,\sigma^{2})$ samples, we start from the general identity

[TABLE]

where $F_{k}(x)=\int_{-\infty}^{x}f_{k}(y)\,dy$ is the CDF of $X_{(k)}$. By expanding the joint Gaussian density and applying the binomial, we have

[TABLE]

with $f(x)$, $F(x)$ the standard normal PDF and CDF. Integrating term‐by‐term yields

[TABLE]

In the regime $n-k\ll n$, the lower‐tail integral $\int_{-\infty}^{0}F_{k}(x)\,dx$ is negligible, and for $x\geq 0$, $F(x)^{\,n-i}$ is exponentially small. We shall approximate the expectation by (see Appendix Appendix C: Reshuffling for a Gaussian Distribution):

[TABLE]

We conclude that for $k\approx n$, the mean position of the kth order position of n Gaussian i.i.d variable is located at:

[TABLE]

In particular, for $k=n-m$ with $m\ll n$, this simplifies to

[TABLE]

This expression captures both the leading $\sigma\sqrt{2\ln n}$ term and finite‐$n$ corrections via the alternating sum. We shall now continue with further approximation of $\sqrt{\ln(n-i)}$: in the regime $i\ll n$, we write

[TABLE]

Since $\tfrac{i}{n}$ is small, $\ln(1-\tfrac{i}{n})=O(\tfrac{i}{n})\ll 1$. Hence we Taylor‐expand $\sqrt{1+x}$ at $x=0$:

[TABLE]

Taking $x=\tfrac{\ln(1-\tfrac{i}{n})}{\ln n}$ gives the one‐term approximation

[TABLE]

A further first‐order Taylor of $\ln(1-x)\approx-x$ then yields

[TABLE]

Substituting into the alternating sum in (121), where $K=n-k$, we obtain

[TABLE]

We now simplify the two key alternating sums. First, observe the partial‐fraction identity

[TABLE]

which follows by evaluating the decomposition $\displaystyle\tfrac{1}{z(z-1)\cdots(z-K)}=\sum_{i}\frac{a_{i}}{z-i}$ at $z=n$ leading to

[TABLE]

Second, for the sum weighted by $i$, note

[TABLE]

Combining these two identities yields the compact approximation

[TABLE]

which simplifies to

[TABLE]

Thus to first order one recovers the familiar $\sigma\sqrt{2\ln n}$ scaling, with a O\bigl{(}(\ln n)^{-1}\bigr{)} correction. We obtain here an analytical expression that does not depend on $k$. To clarify this result, we shall now consider the second-order expansion analysis.

Higher‐order Taylor corrections

To refine the first‐order approximation \sqrt{\ln(n-i)}\approx\sqrt{\ln n}\bigl{(}1-\tfrac{i}{2n\ln n}\bigr{)}, we carry out a second‐order expansion of $\ln(1-x)$ about $x=0$. Recall

[TABLE]

Hence

[TABLE]

Taking square‐roots gives

[TABLE]

Expanding $\sqrt{1+u}=1+\tfrac{u}{2}-\tfrac{u^{2}}{8}+O(u^{3})$ with $u=-\tfrac{i}{n\ln n}-\tfrac{i^{2}}{2n^{2}\ln n}$ yields

[TABLE]

Retaining only terms up to $O(n^{-2}\ln n^{-1})$ gives

[TABLE]

Substituting this into the alternating‐sum formula

[TABLE]

we must evaluate the sums:

[TABLE]

The standard partial‐fraction argument shows

[TABLE]

while a similar index shift gives

[TABLE]

Hence the expanded expectation becomes

[TABLE]

Substituting the closed‐form $S_{0},S_{1},S_{2}$ and simplifying yields

[TABLE]

which refines the leading $\sigma\sqrt{2\ln n}$ term by $O((\ln n)^{-1})$ correction. To conclude the mean position of the $k^{th}$ order position of $n$ Gaussian i.i.d. can be approximated by

[TABLE]

We obtain an approximation for $\mathbb{E}[X_{(k)}]$ that depends only on $n$ (not on $k$) once $k$ is in the extreme tail. In fact, carrying the Taylor expansion of $\ln(1-x)$ to $j$th order shows that

[TABLE]

To evalute the quality of this approximation, we compared, for $n=100$ and $\sigma=1$, the two sequences

[TABLE]

for $k=n,n-1,\dots,n-4$. As shown in Table 1, $\lvert u_{k}-v_{k}\rvert<4\times 10^{-2}$ in all cases, demonstrating that the harmonic‐sum correction yields excellent accuracy for small $j$.

2.4 Practical approximation

Based on the above derivation, we propose the following practical approximation for the expected location of the $k$th order statistic in an i.i.d. sample of size $n$ from $\mathcal{N}(0,\sigma^{2})$:

[TABLE]

Here $H_{n-k}$ is the $(n-k)$th harmonic number, introducing a mild finite-$n$ correction to the leading $\sigma\sqrt{\frac{\pi}{2}\ln n}$ growth. Interestingly, this expression is broadly consistent with the result derived in [29], as for sufficiently large values of $(n-k)$, we have $H_{n-k}\approx\ln(n-k)$.

By symmetry of the centered Gaussian law,

[TABLE]

and more generally, a nonzero mean $\mu$ simply shifts every order statistic by $\mu$:

[TABLE]

Combining (34)–(36) with the definition $\displaystyle T_{\mathrm{shrink}}^{(p)}=\frac{X_{(n-p)}-X_{(p+1)}}{\,X_{(n-p+1)}-X_{(p)}\,}$, one obtains the Gaussian analog of (11):

[TABLE]

This formula holds for any mean $\mu$ and variance $\sigma^{2}$, since both cancel in the ratio. Figure 2 compares the uniform prediction $\tfrac{n-2p-1}{n-2p+1}$ and the Gaussian approximation (37) against Monte Carlo simulations. In the Gaussian case we observe a small, nearly constant bias $\alpha\approx 0.03$. To improve empirical fit, one may therefore use

[TABLE]

In the next section we leverage these analytical shrinkage curves to construct combined likelihood‐ and geometry‐based tests for distinguishing heavy‐tailed (Gaussian) from light‐tailed (uniform) point clouds.

2.5 Shift of the Sample Mean when Removing an Extreme Gaussian Outlier

We now quantify how the sample mean (center of mass) is shifted when we discard the single largest observation from $n$ i.i.d. draws $X_{(1)}\leq\cdots\leq X_{(n)}\sim\mathcal{N}(\mu,\sigma^{2})$. By definition,

[TABLE]

since the sample mean is unbiased.

Trimmed mean after removing the maximum.

Delete $X_{(n)}$ and form the $(n-1)$-point mean $\displaystyle\bar{X}_{-}=\frac{1}{n-1}\sum_{i=1}^{n-1}X_{(i)}.$ Its expectation is

[TABLE]

By symmetry of the centered Gaussian order‐statistic expectations (cf. (35)), $\sum_{i=2}^{n-1}\mathbb{E}[X_{(i)}]=(n-2)\mu$, and from (34) we have $\mathbb{E}[X_{(1)}]\approx\mu-\tfrac{\sigma\sqrt{2\pi\ln n}}{2}$. Therefore

[TABLE]

Net shift.

The expected shift in the center of mass upon removal of the single largest point is thus

[TABLE]

By an identical argument for the smallest point—and by symmetry—the simultaneous removal of both extremes produces no first‐order shift in the mean.

Fig. 2 panels (A)/(B) compare the uniform and Gaussian diameter‐shrinkage curves (with $n=100$, averaged over $10^{4}$ trials) and demonstrates the quality of our analytical approximation. In the next section, we integrate these geometric diagnostics into a robust clustering criterion.

3 Identification of meaningful clusters

As stated in section 1, many spatial clustering algorithms operate under the assumption that clusters are simply regions of high point density separated by areas of lower density, without considering what actually constitutes a meaningful cluster. Clustering methods such as K-means or DBSCAN typically rely on geometric heuristics (e.g. minimizing within‐cluster variance or finding dense neighborhoods) to partition data into groups. They search for geometric proximity or density thresholds, neglecting whether the points are close to each other by coincidence or due to an underlying distribution. However, these methods do not by themselves answer whether a detected “cluster” actually reflects an underlying generative mechanism, or is simply a random fluctuation. To address this, we propose a two‐statistical test that asks: does this one‐dimensional point cloud look more like a uniform distribution, or from a Gaussian distribution? Rejecting the uniform null in favor of a Gaussian alternative provides evidence of a true “hotspot” rather than mere spatial chance. As a result, classical algorithms may identify clusters that are mathematically valid but meaningless or misleading in practical applications.

3.1 Two Competing Models

Starting with $x_{1},\dots,x_{n}$ observed points on the real line, and write $X_{(1)}\leq\cdots\leq X_{(n)}$ for their order statistics. We compare:

[TABLE]

Two natural, complementary approaches are: Likelihood‐Based Model Selection, vs Diameter‐Shrinkage–Based Test that we shall present now.

(a) Likelihood‐Based Model Selection

Uniform model.

Under $\mathcal{U}[a,b]$, each observation has density $\tfrac{1}{b-a}$ when $x_{i}\in[a,b]$, and zero otherwise. Thus the joint likelihood is

[TABLE]

Maximizing over $a,b$ simply forces $a=X_{(1)}$, $b=X_{(n)}$, giving

[TABLE]

Gaussian model.

For $X_{i}\sim\mathcal{N}(\mu,\sigma^{2})$, the likelihood is

[TABLE]

One shows the MLEs are the sample mean and variance, $\hat{\mu}=\bar{x}$, $\hat{\sigma}^{2}=\tfrac{1}{n}\sum(x_{i}-\bar{x})^{2}$, yielding

[TABLE]

Model comparison.

Assuming equal prior weight on $H_{0}$ and $H_{1}$, the posterior odds reduce to the likelihood‐ratio test:

[TABLE]

Equivalently, we can compare $\ln\mathcal{L}_{U}$ vs. $\ln\mathcal{L}_{G}$ using the classical rule: assume a uniform prior over models, the posterior probability of the uniform model is:

[TABLE]

The decision rule is straightforward:

• •

Prefer $H_{0}$ (uniform) if $\ln\mathcal{L}_{U}>\ln\mathcal{L}_{G}$,

• •

Prefer $H_{1}$ (Gaussian) otherwise.

(b) Diameter‐Shrinkage–Based Test

As an alternative, model‐free indicator of tail behavior, we track how the sample diameter

[TABLE]

contracts as we trim $p$ extreme points from each end. Define the successive shrinkage ratio

[TABLE]

Under the uniform model one shows $\mathbb{E}[T_{\mathrm{shrink}}^{(p)}]\approx\tfrac{n-2p-1}{\,n-2p+1\,}$, whereas for a Gaussian sample $\displaystyle\mathbb{E}[T_{\mathrm{shrink}}^{(p)}]\approx\frac{1-\tfrac{H_{p}}{2\ln n}}{1-\tfrac{H_{p-1}}{2\ln n}}$ (see Section 2.3). We derived the approximation:

[TABLE]

The resulting shrinkage profile exhibits a slight increase with $p$, which can be attributed to the heavy tails of the Gaussian distribution. In practical applications, these shrinkage patterns provide a more informative characterization of the underlying distribution than a shrinkage computed at a fixed trimming depth $p$. In practice, we recommend computing $T_{\text{shrink}}^{(p)}$ for several small values of $p$ and comparing the empirical decay pattern against the theoretical curves for each model. This method can be particularly informative when the sample size is moderate and likelihood-based estimates are less stable. We then used this Diameter Shrinkage Statistics to construct a criterion for distinguishing between the Uniform and Gaussian distributions. We shall now summarize our decision criteria.

3.1.1 Decision Rule and Geometric Classifier

We fix here a trimming depth $p\geq 1$ and the observed shrinkage ratios up to step $p$ is defined as \mathbf{T}_{\mathrm{emp}}^{(p)}=\bigl{(}T_{\mathrm{shrink}}^{(1)},\dots,T_{\mathrm{shrink}}^{(p)}\bigr{)}. The expectation vectors $\mathbf{T}_{U}^{(p)}$ and $\mathbf{T}_{G}^{(p)}$ under $H_{0}$ (uniform) and $H_{1}$ (Gaussian), respectively, will be used to classify the sample by comparing Euclidean distances:

[TABLE]

where $\phi_{p}=1$ denotes classification as Gaussian. Figure 3(A–B) illustrates this geometric decision rule.

Monte Carlo Calibration of $p$

To choose an optimal trimming depth $p$, we performed the following simulations: for each $p=1,2,\dots,15$, we generated $N=1000$ samples of size $n=100$ under both

[TABLE]

For each sample, we computed $\mathbf{T}_{\mathrm{emp}}^{(p)}$, applied the rule above, and recorded the classification decision $\phi_{p}$. The empirical accuracy is computed as

[TABLE]

where $y^{(j)}\in\{0,1\}$ is the true model label for the $j$th sample. Figure 3(C) shows $\widehat{\mathrm{Accuracy}}(p)$ vs $p$. Accuracy rises steadily with increasing $p$, reaching a plateau around $p=6$. Beyond this point, further trimming reduces the number of remaining points too sharply, degrading performance. Thus $p=6$ offers an optimal trade–off between sensitivity to tail behavior and statistical sample size.

To conclude, we adopt $p=6$ in subsequent applications. The full algorithm, including a natural confidence measure based on relative distances to the two model curves, is given in the description of Algorithm 1 below:

3.1.2 Empirical Comparison of Likelihood‐Ratio and Shrinkage‐Based Classifiers

To assess the relative robustness of the two approaches, we use a Monte Carlo approach: for each sample size

[TABLE]

we generated:

• •

$N=100$ independent samples of size $n$ from $\mathcal{U}(0,L)$, with the interval length $L$ drawn uniformly from $[0,20]$;

• •

$N=100$ independent samples of size $n$ from $\mathcal{N}(0,\sigma^{2})$, with the standard deviation $\sigma$ drawn uniformly from $[0,20]$.

Table 2 summarizes the overall classification accuracy and AUC (ROC) achieved by each method:

Figure 3D shows the classification accuracy vs $n$. The shrinkage‐based classifier outperforms the likelihood‐ratio test for moderate sample sizes ($20\leq n\leq 60$), where our tail‐trimming approximations remain valid but likelihood estimates are still noisy. For very small $n<20$, neither method is reliable; for large $n>60$, the MLE becomes nearly optimal as the data volume overwhelms the impact of trimming.

A possible interpretation of those results is that if $n<20$, the data is too scarce to be trimmed and the sample size is too low for our approximations to be valid. On the other hand, when $n>60$, the data are large enough for the likelihood to account for the extreme points underlying distribution. To summarize, to classify i.i.d. real-valued samples $x_{1},\dots,x_{n}\in\mathbb{R}$ into a Uniform model or a Gaussian model, depending on the sample size $n$, we recommend the following decision rules:

• •

Moderate sample sizes ($20<n<60$): employ the shrinkage‐based test, which additionally provides a natural confidence metric via distance to the theoretical shrinkage curve.

• •

Other regimes ($n\leq 20$ or $n>60$): default to the classical likelihood‐ratio test, leveraging its asymptotic efficiency.

In the next section, we demonstrate how this hybrid decision rule effectively identifies meaningful clusters in real‐world one‐dimensional spatial data.

3.2 Validation of One-Dimensional Clustering shrinkage method

We assess here our method’s ability to identify statistically meaningful clusters in one dimension by applying it to data ( $N=100$ independent datasets) generated as follows (see Fig. 4-(A):

Sample $n_{A}=10$ “anchor” locations uniformly on the interval $[0,W]$ with $W=10{,}000$. 2. 2.

Generate $n_{B}=1000$ additional points, of which

• •

50% are drawn from $\mathcal{N}(\mu_{i},\sigma^{2})$ with $\sigma=20$, where each $\mu_{i}$ is chosen uniformly from the $n_{A}$ anchor set,

• •

50% are drawn uniformly from $[0,W]$ as background noise.

Each dataset was clustered with DBSCAN (Fig. 4-(B) using $\varepsilon=20$ and $\mathrm{min\_{samples}}=7$, parameters chosen to recover the visually apparent Gaussian clusters. A DBSCAN cluster was labeled “significant” if more than half of its points originated from the Gaussian component. For each significant cluster, we then applied

• •

the combined MLE–shrinkage classifier (Fig. 3), and

• •

the classical likelihood‐ratio test alone.

Over the $N=100$ trials, the hybrid MLE–shrinkage method achieved a balanced accuracy of 0.943, versus 0.920 for the likelihood‐ratio test. This improvement underscores the added robustness and discriminative power conferred by the diameter‐shrinkage statistic—especially notable given that it requires no density fitting or parameter tuning.

4 Conclusion

We have introduced a novel geometric test statistic, based on successive diameter shrinkage, for discriminating between light‐tailed (uniform) and heavy‐tailed (Gaussian) point cloud clustering in one dimension. In contrast to the classical likelihood‐ratio test—which is asymptotically efficient under correct model specification but can suffer in small samples or in the presence of outliers—our shrinkage‐based criterion directly exploits the extreme‐value geometry of the data. This delivers superior discrimination in small‐sample or noisy settings. By monitoring how the sample’s span contracts under successive removal of outliers, it exposes tail behavior without ever fitting a full density. In addition, we derived here analytic approximations for the expected shrinkage under both uniform and Gaussian hypotheses, and demonstrated that these curves separate cleanly even for moderate sample sizes. The resulting procedure has several properties:

• •

Robustness. By focusing on the contraction of the sample span under trimming, the method remains stable under moderate deviations from the assumed density and can account robustly for individual outliers.

• •

Nonparametric character. No explicit density estimation is required; only the order statistics are needed to compute the test statistic.

• •

Extendibility. The same geometric principle may be generalized to higher dimensions via convex‐hull shrinkage or random projections, opening the door to robust cluster validation in multivariate settings.

We used Monte Carlo simulations to confirm that the diameter‐shrinkage test complements maximum‐likelihood methods, achieving superior classification accuracy in small‐sample and noisy regimes while retaining consistency in large‐sample limits. In practice, the two approaches are highly complementary: The MLE‐based likelihood‐ratio test excels when the sample size is large and the model assumptions hold exactly, whereas the shrinkage‐based test can be used when robustness to outliers or moderate deviations is critical. Moreover, the shrinkage framework can generalize to higher dimensions—via convex‐hull trimming, random projections, or other geometric summaries—offering a toolkit for cluster validation and distributional inference. In future work, it would be of interest to study the properties of the statistic in (1) directly, without relying on the approximation in (2), particularly in the case of the Gaussian distribution.

Appendix A. Order‐Statistic Perturbations under Uniform Sampling

In this appendix we derive the exact distribution of the order statistics of $n$ i.i.d. uniform $[0,L]$ samples, and then compute how their span (diameter) and centre‐of‐mass shift when the most extreme points are removed.

Joint density of ordered samples.

Since each of the $n!$ permutations of $(X_{(1)},\dots,X_{(n)})$ is equally likely and each $X_{i}$ has density $1/L$ on $[0,L]$, the joint density of the sorted vector $(X_{(1)},\dots,X_{(n)})$ is simply

[TABLE]

Outside the simplex $0\leq x_{1}\leq\cdots\leq x_{n}\leq L$ the density vanishes, and inside it is constant.

A.1. Marginal density of the $k$th order statistic

To isolate the $k$th statistic $X_{(k)}$, we integrate out all other coordinates. One finds by repeated integration of polynomials that

[TABLE]

for $0\leq x\leq L$. Equivalently, $X_{(k)}/L$ follows a $\mathrm{Beta}(k,n-k+1)$ law. Indeed, a direct computation leads to

[TABLE]

where

[TABLE]

Using that for $a,b\in\mathbb{R}$ and $N\in\mathbb{N}$, we have

[TABLE]

A direct integration leads to

[TABLE]

Similarly,

[TABLE]

Finally, the distribution of $X_{(k)}$ is given by

[TABLE]

A.2. Expected value and higher moments

The closed‐form density immediately yields moments. The first moment is be expressed as:

[TABLE]

which can be computed by using eq. (9) as

[TABLE]

Using the $\beta-$ function defined as:

[TABLE]

We get by using the substitution $t=\frac{x_{k}}{L}$, $dt=\frac{dx_{k}}{L}$

[TABLE]

Finally, using for $x,y\in\mathbb{R}$,

[TABLE]

where

[TABLE]

and for integer $n$, $\Gamma(n)=(n-1)!$, we obtain

[TABLE]

Finally, the expected value of $X_{(k)}$ can be expressed as:

[TABLE]

More generally, for integer $p\geq 1$,

[TABLE]

All these results and computations can be found in the literature [26, 27, 28].

A.3. Trimmed diameter

Define the “$p$–trimmed diameter” by removing the $p$ smallest and $p$ largest points:

[TABLE]

Since $\mathbb{E}[X_{(m)}]=L\,\tfrac{m}{n+1}$, it follows that

[TABLE]

For $p=0$, this result can also be obtained as follows: first, the joint distribution of $X_{(1)},X_{(n)}$ is given by

[TABLE]

By using (64), we have, if $0\leq x_{1}\leq x_{n}\leq L$:

[TABLE]

Hence, the joint distribution of $X_{(1)},X_{(n)}$ can be expressed as:

[TABLE]

Therefore,

[TABLE]

A.4. Shrinkage ratio after trimming

We compute here the expected value of the random variable

[TABLE]

As was done previously, the first step is to find the joint distribution of $X_{(1)},X_{n-1},X_{(n)}$.

[TABLE]

By using (64), we have, if $0\leq x_{1}\leq x_{n-1}\leq x_{n}\leq L$:

[TABLE]

Hence, the joint distribution of $X_{(1)},X_{(n)}$ can be expressed as:

[TABLE]

Thus,

[TABLE]

We shall use the notation

[TABLE]

First, we consider

[TABLE]

Then,

[TABLE]

By integrating by parts:

[TABLE]

Finally,

[TABLE]

By merging (90) and (77), we have:

[TABLE]

Therefore, the expected value of segment length evolution ratio after removing of $X_{(n)}$ can be expressed by:

[TABLE]

It is interesting to further notice that

[TABLE]

which somehow justifies the approximation (2), stating that:

[TABLE]

Appendix B: Alternative Elementary Derivation of $\mathbb{E}[\max_{1\leq i\leq n}|S_{i}|]$ Without the Chu–Tanner Fit

We recall that $X_{1},...,X_{n}\stackrel{{\scriptstyle iid}}{{\sim}}\mathcal{N}(0,\sigma^{2})$ are defined by $M_{n}=\max_{1\leq i\leq n}|X_{i}|$. We shall show here that the expected maximum satisfies:

[TABLE]

where $\gamma$ is Euler’s constant. The expectation decomposes as:

[TABLE]

where $a_{n}=\sigma\sqrt{2\log n}$. We shall now decompose the integral. This follows from:

[TABLE]

To study $J_{n}$, we use $\epsilon_{n}=\frac{\sigma\log\log n}{\sqrt{2\log n}}$ and split the integral $J_{n}$ into:

[TABLE]

To analyse $J_{n}^{(1)}$ in th Bulk Region), we have for $r\leq a_{n}-\epsilon_{n}$:

[TABLE]

Using the erf asymptotic expansion:

[TABLE]

where $\delta_{n}=\frac{\log\log n}{2\sqrt{\log n}}$. We expand the exponent:

[TABLE]

Thus:

[TABLE]

Therefore:

[TABLE]

Raising to the $n$-th power:

[TABLE]

Thus:

[TABLE]

We now analyse $J_{n}^{(2)}$ (Boundary Layer): for $r\in[a_{n}-\epsilon_{n},a_{n}]$, we get

[TABLE]

The erf term becomes:

[TABLE]

Simplifying:

[TABLE]

Thus:

[TABLE]

Using $(1-x/n)^{n}\approx e^{-x}$:

[TABLE]

For large $n$, this becomes:

[TABLE]

We get the final result:

[TABLE]

Thus:

[TABLE]

We now analyze the tail contribution to the expected maximum:

[TABLE]

where $a_{n}=\sigma\sqrt{2\log n}$. changing Variables

[TABLE]

The integral transforms to:

[TABLE]

For $x\gg 1$:

[TABLE]

Let $x=\sqrt{\log n}+\frac{y}{2\sqrt{\log n}}$. Then:

[TABLE]

Thus:

[TABLE]

and:

[TABLE]

The survival function becomes:

[TABLE]

Using $(1-\frac{c}{n})^{n}\approx e^{-c}$:

[TABLE]

The integral decomposes as:

[TABLE]

Region 1: $0\leq y\leq\sqrt{\log n}$

[TABLE]

Integrating:

[TABLE]

Region 2: $y>\sqrt{\log n}$

[TABLE]

The dominant term comes from:

[TABLE]

after appropriate rescaling. Combining all terms:

[TABLE]

Appendix C: Reshuffling for a Gaussian Distribution

We recall here the distribution for $S_{1},\dots,S_{n}$ i.i.d following a Gaussian law $\mathcal{N}(0,\sigma^{2})$. Since there are $n!$ possible permutations and the density of a random variable following such Gaussian law is $\displaystyle\frac{e^{-\frac{x^{2}}{2\sigma^{2}}}}{\sigma\sqrt{2\pi}}dx$, the probability density function (pdf) of the order statistics $X_{(1)},\dots,X_{(n)}$ is

[TABLE]

B.1. Expected value of $X_{(k)}$

We now compute the mean associated to the $X_{(k)}-$variable. It can be expressed as

[TABLE]

where $F_{k}$ is the cumulative distribution function of the $X_{(k)}-$variable

[TABLE]

For instance, for $k=n$, using eq. (25):

[TABLE]

By using the substitution $u=-y$, $du=-dy$:

[TABLE]

We can then make the approximation that for any $x\leq 0$, $F_{n}(x)<<1-$. For the first integral of (100), by using the same substitutions as in subsubsection 2.3.1, we get

[TABLE]

Using the convergence (see subsubsection 2.3.1), we obtain the leading order term

[TABLE]

The second integral of (100) can be computed similarly:

[TABLE]

Since

[TABLE]

dominated convergence yields

[TABLE]

We therefore obtain the expected value of the maximum among $n$ Gaussian variables:

[TABLE]

For any $k$,we rewrite the density function $f_{k}$ as

[TABLE]

where $\displaystyle f(x)=\frac{e^{-\frac{x^{2}}{2\sigma^{2}}}}{\sigma\sqrt{2\pi}}$ and $\displaystyle F(x)=\frac{\text{erfc}(-\frac{x}{\sigma\sqrt{2}})}{2}$ are respectively the density and cumulative distribution functions of a Gaussian law $\mathcal{N}(0,\sigma^{2}).$ We can rewrite $f_{k}$ as

[TABLE]

This is equivalent to the cumulative form:

[TABLE]

Finally,

[TABLE]

We conclude that for $n-k\ll n$ and $i\ll n$, we can use similar arguments than for $k=n$ to prove that $\displaystyle\int_{-\infty}^{0}F(x)^{n-i}\longrightarrow\ 0$. Hence,

[TABLE]

Finally, we can conclude that the expected value of $X_{(k)}$ when $n-k<<n$ can be approximated by

[TABLE]

5 Acknowledgements

We thank S. Majumdar for discussions on this manuscript and pointing out his recent book as a reference.