Bivariate distributions with ordered marginals

TL;DR

This paper characterizes dependency structures between two stochastically ordered variables using copulas, providing extremal measures, explicit joint distributions, and extensions to multivariate and partially ordered spaces.

Contribution

It offers a comprehensive characterization of copulas compatible with stochastic order, including extremal measures and maximum entropy distributions, with multivariate extensions.

Findings

Closed-form extremal Kendall's tau and Spearman's rho

Explicit maximum entropy joint distribution

Extensions to multivariate and partially ordered spaces

Abstract

This paper provides a characterization of all possible dependency structures between two stochastically ordered random variables. The answer is given in terms of copulas that are compatible with the stochastic order and the marginal distributions. The extremal values for Kendall's $\tau$ and Spearman's $\rho$ for all these copulas are given in closed form. We also find an explicit form for the joint distribution with the maximal entropy. A multivariate extension and a generalization to random elements in partially ordered spaces are also provided.

Equations

$$G(x):=\frac{1}{2}(F_1(x)+F_2(x)),x\in \mathbb{R}.$$

$$\mathbb{P}(V_1\le x,V_2\le x)=\mathbb{P}(X_1\le x,X_2\le x)=\mathbb{P}(X_1\le x).$$

$$\mathbb{P}(X_1\le x)$$

$$\mathbb{P}(X_2\le x)$$

$$=2G(x)-F_1(x)=F_2(x).\text{qed}$$

$$F(x_1,x_2)=C(F_1(x_1),F_2(x_2))$$

$$F(x_1,x_2)=\{\begin{array}{ll}{F}_1(x_1),& x_1\le x_2,\\ 2\tilde{C}(G(x_1),G(x_2))-F_1(x_2),& x_1>x_2,\end{array}$$

$$\tilde{C}(G(x),G(x))=F_1(x),x\in \mathbb{R}.$$

$$\tilde{C}(G(x),G(y))$$

$$F_1(x)$$

$$F^{-}(t):=\inf \{x:F(x)\ge t\}\in [-\infty ,+\infty ],t\in [0,1],$$

$$F\circ F^{-}(t)=t$$

$$D:=F_1\circ G^{-},$$

$${\delta }^{-}(x+)=\underset{y\downarrow x}{\lim }{\delta }^{-}(y),{\delta }^{-}(1+)=1.$$

$$0\le y-x=\delta ({\delta }^{-}(y+))-\delta ({\delta }^{-}(x+))\le 2({\delta }^{-}(y+)-{\delta }^{-}(x+)).$$

$$G^{-}(t)$$

$$=\delta (\inf \{{\delta }^{-}(x+):{\delta }^{-}(x+)\ge t\})=\delta (t).\text{qed}$$

$$F_1(x_2)=\tilde{C}(G(x_2),G(x_2))\le \tilde{C}(G(x_1),G(x_2))\le \tilde{C}(1,G(x_2))=G(x_2),$$

$$\tilde{C}((p+q)/2,(p+q)/2)=p.$$

$$F(x_1,x_2)=2\tilde{C}(x_1/2,x_2/2).$$

$$B_{\delta }(u,v)\le \tilde{C}(u,v),u,v\in [0,1],$$

$$B_{\delta }(u,v):=\min \{u,v\}-\inf \{t-\delta (t):t\in [\min \{u,v\},\max \{u,v\}]\},u,v\in [0,1],$$

$$H(x):=F_2(x)-F_1(x),x\in \mathbb{R}.$$

$$L(x_1,x_2)\le F(x_1,x_2)\le \min \{F_1(x_1),F_2(x_2)\},x_1,x_2\in \mathbb{R},$$

$$L(x_1,x_2):=\{\begin{array}{ll}{F}_1(x_1),& x_1\le x_2,\\ F_2(x_2)-{\inf }_{x_2\le s\le x_1}H(s),& x_1>x_2,\end{array}$$

$${\delta }_G(x)=\max \{D(x^{-}),D(x^{+})-2(x^{+}-x)\},$$

$$F(x_1,x_2)\ge 2B_{{\delta }_G}(G(x_1),G(x_2))-F_1(x_2)=2(G(x_2)-\inf \{t-{\delta }_G(t):t\in [G(x_2),G(x_1)]\})-F_1(x_2).$$

$$t-{\delta }_G(t)=\min \{t^{-}-D(t^{-})+t-t^{-},t^{+}-D(t^{+})+t^{+}-t\},$$

$$F(x_1,x_2)$$

$$L(x_1,x_2)=\{\begin{array}{ll}{F}_1(x_1),& x_1\le x_2,\\ F_1(x_1)-\min \{F_1(x_1)-F_1(x_2),F_2(x_1)-F_2(x_2)\},& x_1>x_2.\end{array}$$

$$F_2(x_2)-\underset{x_2\le s\le x_1}{\inf }H(s)$$

$$=F_1(x_1)-\min \{F_2(x_1)-F_1(x_1)-F_2(x_2)+F_1(x_1),F_2(x_2)-F_1(x_2)-F_2(x_2)+F_1(x_1)\},$$

$$H(x)=F_2(x)-F_1(x)=\{\begin{array}{ll}x,& x\in [0,1],\\ 2-x,& x\in [1,2],\\ 0,& \text{otherwise},\end{array}$$

Full text

1 Introduction

Let $X_{1}$ and $X_{2}$ be two random variables, such that $X_{1}$ is stochastically larger than $X_{2}$. This means that $F_{1}(x)\leq F_{2}(x)$, $x\in\mathbb{R}$, for their cumulative distribution functions (cdfs) $F_{1}$ and $F_{2}$, respectively. It is well known that this is the case if and only if $X_{1}$ and $X_{2}$ can be realized on the same probability space, so that $X_{1}\geq X_{2}$ almost surely. The objective of this paper is to characterize all random vectors $(X_{1},X_{2})$ such that $X_{1}$ and $X_{2}$ have given cdfs and $\mathbb{P}(X_{1}\geq X_{2})=1$.

As a first observation, we establish a representation of joint distributions of ordered random variables as distributions of the order statistics sampled from an exchangeable bivariate law.

Theorem 1**.**

A random vector $(X_{1},X_{2})$ with marginal cdfs $F_{1}$ and $F_{2}$ satisfies $\mathbb{P}(X_{1}\geq X_{2})=1$ if and only if $X_{1}=\max(V_{1},V_{2})$ and $X_{2}=\min(V_{1},V_{2})$ for a random vector $(V_{1},V_{2})$ with exchangeable components and such that $\mathbb{P}(V_{1}\leq x,V_{2}\leq x)=F_{1}(x)$ and $\mathbb{P}(V_{i}\leq x)=G(x)$ for all $x$, $i\in\{1,2\}$, where

[TABLE]

Proof*.*

The vector $(V_{1},V_{2})$ obtained as the random permutation of $(X_{1},X_{2})$ is exchangeable and its marginal distributions are $G$. Furthermore,

[TABLE]

Conversely, if $X_{1}$ and $X_{2}$ are order statistics from $(V_{1},V_{2})$, then $X_{1}\geq X_{2}$ a.s., and

[TABLE]

This theorem complements already known results deriving the distribution of order statistics from general multivariate laws, see [1, 2, 3].

If the supports of $X_{1}$ and $X_{2}$ are disjoint intervals, then any dependency structure between them is possible. Otherwise, restrictions are necessary, e.g. $X_{1}$ and $X_{2}$ cannot be independent. In Section 2, we give a complete description of the joint distribution of $X_{1}$ and $X_{2}$. This description is given in terms of copulas and their diagonal sections. In Section 3, we identify bounds on the joint distribution of $(X_{1},X_{2})$ in terms of concordance ordering. Then in Section 4, we determine the smallest possible nonparametric correlation coefficients. The joint distribution of $(X_{1},X_{2})$ with the maximal entropy is found in Section 5, followed by examples in Section 6. A multivariate extension and a generalization to random elements in partially ordered spaces are presented in Section 7.

2 Characterization of stochastically ordered copulas

A (bivariate) copula $C:[0,1]^{2}\to[0,1]$ is the cdf of a random vector $(U_{1},U_{2})$ with standard uniformly distributed marginals. The joint cdf of each random vector $(X_{1},X_{2})$ can be written as

[TABLE]

for a copula $C$ with $F_{1}$ and $F_{2}$ being the marginal cdfs. A copula $C$ is called symmetric on a set $A\subset[0,1]$ if $C(u,v)=C(v,u)$ for all $u,v\in A$. For $A=[0,1]$, symmetry of the copula is equivalent to the pair $(U_{1},U_{2})$ being exchangeable.

The following theorem provides a characterization of all dependence structures that are compatible with the stochastic ordering of the marginals.

Theorem 2**.**

Let $(X_{1},X_{2})$ be a random vector with marginals $X_{1}$ and $X_{2}$ having cdfs $F_{1}$ and $F_{2}$, respectively. Then $\mathbb{P}(X_{1}\geq X_{2})=1$ if and only if $F_{1}(x)\leq F_{2}(x)$ for all $x\in\mathbb{R}$ and the joint cdf of $(X_{1},X_{2})$ is given by

[TABLE]

for all $x_{1},x_{2}\in\mathbb{R}$, where $G$ is given by (1) and $\tilde{C}$ is a symmetric copula on the range of $G$ such that

[TABLE]

Proof*.*

Sufficiency. Let $(V_{1},V_{2})$ be distributed according to the symmetric bivariate cdf $\tilde{C}(G(x_{1}),G(x_{2}))$. By construction, $V_{1}$ and $V_{2}$ are identically distributed with cdf $G$. Furthermore, $\mathbb{P}(V_{1}\leq x,V_{2}\leq x)=\tilde{C}(G(x),G(x))=F_{1}(x)$. The distribution of $(\tilde{X}_{1},\tilde{X}_{2})=(\max\{V_{1},V_{2}\},\min\{V_{1},V_{2}\})$ is given by (2), so sufficiency follows from Theorem 1 because $\mathbb{P}(\tilde{X}_{1}\geq\tilde{X_{2}})=\mathbb{P}(X_{1}\geq X_{2})$.

Necessity. Let $(V_{1},V_{2})$ be as in Theorem 1. Then, any copula $\tilde{C}$ of $(V_{1},V_{2})$ satisfies

[TABLE]

for all $x,y\in\mathbb{R}$. ∎

Diagonal sections of copulas, i.e. the functions that arise as $\delta(t)=C(t,t)$, $t\in[0,1]$, for some copula $C$ are characterized by the following properties, see [4].

Definition 1**.**

A function $\delta:[0,1]\to[0,1]$ is a diagonal section if

(D1)

$\delta(0)=0$, $\delta(1)=1$;

(D2)

it is increasing;

(D3)

$|\delta(t)-\delta(s)|\leq 2|t-s|$ for all $t,s\in[0,1]$;

(D4)

$\delta(t)\leq t$ for all $t\in[0,1]$.

For an increasing function $F:\mathbb{R}\to[0,1]$, denote by

[TABLE]

the generalized inverse of $F$, where $\inf\varnothing=+\infty$. Note that

[TABLE]

for all $t$ from the range of $F$, see [5, Proposition 2.3(4)]. For notational convenience, we set $F(-\infty)=0$, $F(\infty)=1$.

The following result follows from the representation (3) of the diagonal section of the copula $\tilde{C}$ and identity (4).

Corollary 3**.**

Let $G$ be given by (1). The function

[TABLE]

given by the composition of $F_{1}$ and the generalized inverse of $G$, is the restriction of a diagonal section to the range of $G$.

We can also provide a converse to Corollary 3.

Proposition 4**.**

Let $\delta$ be a diagonal section. Then there are cdfs $F_{1}$ and $F_{2}$ such that $F_{1}\leq F_{2}$ and $\delta=F_{1}\circ G^{-}$ with $G$ defined at (1).

Proof*.*

We can extend $\delta$ to an increasing function on $\mathbb{R}$ with range $[0,1]$. Its generalized inverse $\delta^{-}$ is left-continuous. For $x\in[0,1)$, we define

[TABLE]

The function $x\mapsto\delta^{-}(x+)$ is increasing and right-continuous, $\delta(\delta^{-}(x+))=x$, and $\delta^{-}(x+)\geq\delta^{-}(x)$ for all $x\in[0,1]$. Let $x\leq y$, $x,y\in[0,1]$. Then, by (D3),

[TABLE]

Set $F_{1}(x)=x$ for $x\in[0,1]$ and $F_{2}(x)=2\delta^{-}(x+)-x$ for $x\in[0,1]$. The function $F_{2}$ is a cdf by the above arguments. By (D4), we obtain $F_{1}\leq F_{2}$. We have $G(x)=\delta^{-}(x+)$, $x\in[0,1]$. It remains to be checked that $G^{-}=\delta$. The function $\delta$ is constant on $[\delta^{-}(x),\delta^{-}(x+)]$ for any $x\in[0,1]$. This implies, for any $t\in(0,1]$,

[TABLE]

Equation (3) specifies the diagonal section $\tilde{C}(t,t)$ of the copula $\tilde{C}$ for all $t$ from the range of $G$. If both $X_{1}$ and $X_{2}$ are non-atomic, then this range is $[0,1]$ and so the diagonal section of $\tilde{C}$ is uniquely specified.

Example 1**.**

An Archimedean generator is a decreasing convex function $\psi:[0,\infty)\to[0,1]$ with $\psi(0)=1$ and $\lim_{x\to\infty}\psi(x)=0$, see [6, Theorem 6.3.2]. Note that we define an Archimedean generator following [7]. For the Archimedean copula $C(u,v)=\psi(\psi^{-}(v)+\psi^{-}(u))$, $u,v\in[0,1]$, with generator $\psi$, the diagonal section is $\delta(t)=\psi(2\psi^{-}(t))$.

There are many parametric families of Archimedean generators. For example, the Gumbel family of copulas is generated by $\psi_{\theta}(t)=\exp(-t^{1/\theta})$, $\theta\in[1,\infty)$. Then, the cdf constructed in the proof of Proposition 4 is $F_{2}(t)=2t^{a}-t$, $t\in[0,1]$, with $a=2^{-1/\theta}$.

Example 2** (Identical distributions).**

If $X_{1}$ and $X_{2}$ are identically distributed, then $F_{1}=F_{2}=G$. In this case, the diagonal section of $\tilde{C}$ in Theorem 2 is given by $D(t)=t$ for all $t$ from the range of $G$. For $x_{1}>x_{2}$, by (3),

[TABLE]

hence, $\tilde{C}(G(x_{1}),G(x_{2}))=F_{1}(x_{2})$. Therefore, by (2), the joint law of $(X_{1},X_{2})$ satisfies $F(x_{1},x_{2})=F_{1}(\min\{x_{1},x_{2}\})$, meaning that $X_{1}=X_{2}$ almost surely.

Example 3** (Discrete distributions).**

Assume that $X_{1}$ and $X_{2}$ have discrete distributions, say supported on $\{0,1\}$ with masses $p,1-p$ and $q,1-q$, respectively, and such that $p\leq q$. The range of $G$ is $\{0,(p+q)/2,1\}$. The condition (3) on $\tilde{C}$ in Theorem  2 is

[TABLE]

While there are clearly many copulas that satisfy this constraint, the condition is sufficient to uniquely determine the joint law of $(X_{1},X_{2})$. By (2), $\mathbb{P}(X_{1}=1,X_{2}=0)=2\tilde{C}(1,(p+q)/2)-2p=q-p$.

Example 4** (Disjoint supports).**

Assume that $X_{1}$ is uniformly distributed on $[1,2]$ and $X_{2}$ on $[0,1]$. In this case, all kinds of dependency structures between $X_{1}$ and $X_{2}$ are allowed. For $x_{1}\in[1,2]$ and $x_{2}\in[0,1]$, (2) yields that

[TABLE]

As prescribed by (3), $\tilde{C}(t,t)=0$ for $t\in[0,1/2]$ and $\tilde{C}(t,t)=2t-1$ for $t\in(1/2,1]$. This is the diagonal section of the Fréchet-Hoeffding lower bound.

It is not a contradiction that any copula $C$ yields a possible bivariate law $F(x_{1},x_{2})=C(F_{1}(x_{1}),F_{2}(x_{2}))$ of $(X_{1},X_{2})$ such that $X_{1}\geq X_{2}$ almost surely, but in the representation of Theorem 2, there are restrictions on the diagonal of the symmetric copula $\tilde{C}$. The copula $\tilde{C}$ is the copula of the random permutation $(V_{1},V_{2})$ of $(X_{1},X_{2})$, and as such it cannot put any mass on the squares $[0,1/2]^{2}$ or $[1/2,1]^{2}$.

3 Pointwise bounds on the joint cdf

By Theorem 2, the range of all possible bivariate cdfs of random vectors $(X_{1},X_{2})$ with given marginals $F_{1}$ and $F_{2}$ and such that $X_{1}\geq X_{2}$ a.s. depends on the choice of a symmetric copula $\tilde{C}$ satisfying (3), equivalently, having the diagonal section (5) on the range of $G$. For a general diagonal section $\delta$, the following result holds.

Theorem 5** ([8, 9]).**

Each copula $\tilde{C}$ with diagonal section $\delta$ satisfies

[TABLE]

where

[TABLE]

is the Bertino copula. The copula $B_{\delta}$ has diagonal section $\delta$.

Denote

[TABLE]

Theorem 6**.**

Each random vector $(X_{1},X_{2})$ with marginal cdfs $F_{1}$ and $F_{2}$, and such that $X_{1}\geq X_{2}$ a.s., has a joint cdf $F$ satisfying

[TABLE]

where both bounds are attained, and

[TABLE]

Proof*.*

The upper bound in (8) is the Fréchet–Hoeffding one; it corresponds to complete dependence between $X_{1}$ and $X_{2}$, so that $X_{1}=F_{1}^{-}(U)$ and $X_{2}=F_{2}^{-}(U)$ for a standard uniformly distributed random variable $U$.

For the lower bound, let $\delta$ be a diagonal section which is equal to $D$ at (5) on the range $R_{G}$ of $G$. We continuously extend $D$ to the closure $\operatorname{cl}(R_{G})$ of $R_{G}$. Continuity of $\delta$ implies that $\delta$ is equal $D$ on $\operatorname{cl}(R_{G})$. The function $\delta$ is bounded below by $\delta_{G}$ defined as

[TABLE]

where $x^{-}=\sup R_{G}\cap[0,x]$ and $x^{+}=\inf R_{G}\cap[x,1]$. This function is itself a diagonal section which is equal to $D$ on $\operatorname{cl}(R_{G})$. For all $u,v\in[0,1]$ it holds that $B_{\delta}(u,v)\geq B_{\delta_{G}}(u,v)$. Therefore, Theorem 5 and (2) imply for $x_{1}>x_{2}$,

[TABLE]

Since

[TABLE]

we can restrict the infimum in (10) to $R_{G}$. Hence,

[TABLE]

The last equality holds because $G^{-}\circ G(x)\leq x$ always holds (see [5]) and $G^{-}\circ G(x)<x$ only happens if there is an $\varepsilon>0$ such that $G$ is constant on $(x-\varepsilon,x]$. But if $G$ is constant on some interval, then $F_{1}$ is necessarily also constant on this interval. ∎

The lower bound in (8) corresponds to the Bertino copula and so yields the least possible dependence between $X_{1}$ and $X_{2}$. If the function $H$ at (7) is unimodal, this corresponds to the assumption that the Bertino copula (6) is simple, compare [10]. The unimodality condition (which also appears in Theorem 10) applies in many examples, and simplifies the structure of the distribution $L$ considerably.

Corollary 7**.**

Assume that the function $H$ is unimodal, that is, $H$ increases on $(-\infty,r]$ and decreases on $[r,\infty)$ for some $r$. Then

[TABLE]

Proof*.*

Let $x_{1}>x_{2}$. By the unimodality, the infimum of $H$ over $[x_{2},x_{1}]$ is attained at one of the end-points $x_{2}$ or $x_{1}$. Therefore,

[TABLE]

which yields (11). ∎

Example 5** (Disjoint supports – Example 4 continued).**

We assume that $X_{1}$ is uniformly distributed on $[1,2]$ and $X_{2}$ on $[0,1]$. Then,

[TABLE]

which is clearly unimodal. Therefore, by Corollary 7, and for $x_{1},x_{2}\in[0,2]$

[TABLE]

which corresponds to choosing the Fréchet-Hoeffding lower bound as the dependence structure for $(X_{1},X_{2})$.

As shown by Rogers in [11], if $H$ is unimodal, the distribution given by (11) maximizes the payoff (or transportation cost) $\mathbb{E}\phi(|X_{1}-X_{2}|)$ over all strictly convex decreasing functions $\phi:\mathbb{R}_{+}\mapsto\mathbb{R}_{+}$. Without unimodality assumption, the joint distribution maximizing the payoff is given by

[TABLE]

This joint distribution satisfies $L(x_{1},x_{2})\leq P(x_{1},x_{2})\leq\min\{F_{1}(x_{1}),F_{2}(x_{2})\}$; it provides the joint distribution with the largest mass concentrated on the diagonal, see [12, Th. 7.2.6].

The following result concerns the support of the random vector with distribution $L$ in the case when the cdfs $F_{1}$ and $F_{2}$ are continuous. In the general case, the support of $L$ is more intricate to describe.

Lemma 8**.**

Suppose that $F_{1}$ and $F_{2}$ are continuous. The support of the distribution $L$ given at (9) is the set

[TABLE]

where $S_{1}$ and $S_{2}$ are the supports of the distributions $F_{1}$ and $F_{2}$, respectively, and

[TABLE]

Here, $\partial S_{i}$, denotes the topological boundary of $S_{i}$, $i=1,2$.

Proof*.*

Let $(X_{1},X_{2})$ have distribution $L$. Let $x\in\mathbb{R}$ and $\varepsilon>0$. Then,

[TABLE]

The right hand side can only be strictly positive if

[TABLE]

which is the case whenever $\mathbb{P}(X_{1}\in(x-\varepsilon,x+\varepsilon])>0$ and $\mathbb{P}(X_{2}\in(x-\varepsilon,x+\varepsilon])>0$. Thus, only points $(x,x)$ with $x$ belonging to $S_{1}$ and $S_{2}$ may be in the diagonal parts of the support of $L$. If $x\in T$ and $\varepsilon>0$ is small enough, then

[TABLE]

so $(x,x)$ cannot belong to the support of $L$.

Conversely, assume that $x\in S_{1}\cap S_{2}$. Since $H$ is continuous, the infimum in (15) is attained at some $s_{0}(\varepsilon)\in[x-\varepsilon,x+\varepsilon]$, if $s_{0}(\varepsilon)<x$, then $x\in(s_{0}(\varepsilon),x+\varepsilon)$, hence $F_{1}(x+\varepsilon)-F_{1}(s_{0}(\varepsilon))>0$ because $x\in S_{1}$. One can argue analogously if $s_{0}(\varepsilon)>x$. If $s_{0}(\varepsilon)=x$, then we distinguish two cases. If $x\in\operatorname{int}S_{1}$ or $x\in\operatorname{int}S_{2}$, then one can argue as previously. Here $\operatorname{int}(S_{i})$ denotes the interior of $S_{i}$, $i\in\{1,2\}$. If $x\in\partial S_{1}\cap\partial S_{2}$ and

[TABLE]

then $x\in T$ which yields the claim concerning the diagonal part of the support of $L$.

Now assume that $x_{2}<x_{1}$ and $0<\varepsilon<(x_{1}-x_{2})/2$. Then

[TABLE]

where

[TABLE]

The probability in (16) is strictly positive if and only if $a\leq c<b$ or $c\leq a<b$. The point $(x_{1},x_{2})$ belongs to the support of $L$ if and only if $\max\{a,c\}<b$ for all $\varepsilon>0$ small enough. Letting $\varepsilon$ converge to zero, we find that a necessary condition is that

[TABLE]

hence $H(x_{1})=H(x_{2})$. It is not hard to check that the conditions on $x_{2}$ and $x_{1}$ in $A$ are necessary and sufficient to ensure that $\max\{a,c\}<b$ is fulfilled for all $\varepsilon>0$ small enough. ∎

The set $A$ from (8) is illustrated in the top-left panel of Fig. 1 using points sampled from $L$.

Example 6** (Disjoint supports – Example 4 continued).**

We assume that $X_{1}$ is uniformly distributed on $[1,2]$ and $X_{2}$ on $[0,1]$. The function $H$ and the distribution $L$ are given at (12) and (13), respectively. The support of $L$ is given by

[TABLE]

This follows from Example 5 or Lemma 8. The set $A$ at (8) consists of three parts. The first set is $\{(2-x_{2},x_{2})\in\mathbb{R}^{2}:\;x_{2}\in[0,1)\}$, the second set is empty, and the third set is $\{(x,x)\in\mathbb{R}^{2}:\;x=1\}$ because $T$ is the empty set.

4 Nonparametric correlation coefficients

Dependence measures quantitatively summarize the degree of dependence between two random variables $X_{1}$ and $X_{2}$. Kendall’s tau and Spearman’s rho are arguably the two most well-known measures of association whose sample versions are purely based on ranks. If the marginal distributions of $X_{1}$ and $X_{2}$ are continuous then the population versions of Kendall’s tau and Spearman’s rho only depend on the copula of $(X_{1},X_{2})$. In this section, we assume that both, $F_{1}$ and $F_{2}$ are continuous, and hence the copula of $(X_{1},X_{2})$ is uniquely defined. We refer the reader to [13] for details concerning problems that arise in the case of arbitrary marginal distributions.

Kendall’s tau of a copula $C$ is given by

[TABLE]

and, Spearman’s rho is given by

[TABLE]

These two correlation coefficients are monotonic with respect to pointwise, or, concordance ordering of copulas [13]. They take the value one for the joint cdf given by the upper bound in (8). For the copula $C(u,v):=L(F_{1}^{-}(u),F_{2}^{-}(v))$ with $L$ given by (9), these dependence measures attain their lowest values calculated as follows.

Theorem 9**.**

Suppose that $F_{1}$ and $F_{2}$ are continuous. The smallest possible Kendall’s tau of $(X_{1},X_{2})$ that satisfies the conditions of Theorem 2 is

[TABLE]

Proof*.*

Writing $C(u,v)=L(F_{1}^{-}(u),F_{2}^{-}(v))$ yields that

[TABLE]

where $(X_{1},X_{2})$ has cdf $L$ given by (9). By Lemma 8, the support of $(X_{1},X_{2})$ is given by the set $A$ at (8). On the set $A$,

[TABLE]

hence the result. ∎

Theorem 10**.**

Suppose that $X_{1}$ and $X_{2}$ have continuous cdfs $F_{1}$ and $F_{2}$ with the same support $[x^{L},x^{U}]$ and that the function $H(s)=F_{2}(s)-F_{1}(s)$ from (7) is unimodal, strictly increases on $(x^{L},r]$ and strictly decreases on $[r,x^{U})$ for some $r$. Then the smallest possible Spearman’s rho of $(X_{1},X_{2})$ satisfying the conditions of Theorem 2 is

[TABLE]

where $F_{2}(t(s))-F_{1}(t(s))=F_{2}(s)-F_{1}(s)$ and $t(s)<s$ for $s\in(r,x^{U}]$ and $t(r)=r$.

Proof*.*

Spearman’s rho is given by

[TABLE]

The set $A$ from (8), consists of 3 pieces: $\{(s,s):\;s\in(x^{L},r]\}$ with the push-forward of $F_{1}(s)$, $s\in(x^{L},r]$; $\{(s,s):\;s\in(r,x^{U})\}$ with the distribution being the image of the measure on $(r,x^{U})$ with push-forward of $F_{2}(s)-F_{2}(r)$; and $\{(s,t(s)):\;s\in(r,x^{U})\}$ with the distribution being the push-forward of $H(r)-H(s)$. The push-forward is the image of the measure on the line by the specified map, e.g., the third part if the image of the measure $\mu$ on $(r,x^{U})$ with $\mu((r,s])=H(r)-H(s)$ under the map $s\mapsto(s,t(s))$.

The result is obtained by splitting the above expectation into these 3 parts. ∎

5 Maximum entropy distributions

We assume that both $X_{1}$ and $X_{2}$ have full supports on a (possibly infinite) interval $[x^{L},x^{U}]$ and that their cdfs $F_{1}$ and $F_{2}$ are absolutely continuous with densities $f_{1}$ and $f_{2}$. Amongst all joint absolutely continuous laws $F$ of $(X_{1},X_{2})$ with given marginals $F_{1}$ and $F_{2}$ and such that $X_{1}\geq X_{2}$ a.s., we characterize those maximizing the differential entropy (see [14, Ch. 8]) given by

[TABLE]

These copulas correspond to the least informative (most random) joint distributions, equivalently, to the distributions minimizing the Kullback–Leibler divergence with respect to the uniform distribution. We use the common convention $0\log 0=0$. Independently of our work, maximum entropy distributions of order statistics in the multivariate case have been studied in [15].

Note that the function $G$ from (1) is absolutely continuous with density $g=(f_{1}+f_{2})/2$. By Theorem 2, the joint law $F$ of $(X_{1},X_{2})$ is absolutely continuous if and only if the associated symmetric copula $\tilde{C}$ is absolutely continuous. We denote its density by $\tilde{c}$. By the symmetry of $\tilde{C}$,

[TABLE]

Therefore, maximizing $\operatorname{ent}(F)$ over all $F$ is equivalent to maximizing $\operatorname{ent}(\tilde{C})$ over all symmetric copulas $\tilde{C}$ with diagonal section $D=F_{1}\circ G^{-}$. Note that the smallest entropy $-\infty$ arises as the limit by considering absolutely continuous distributions approximating the distribution of $X_{1}=F_{1}^{-}(U)$ and $X_{2}=F_{2}^{-}(U)$ for a uniformly distributed $U$.

Butucea et al. [16] characterize the maximum entropy copula with a given diagonal section $\delta$. We recall some of their notation in order to be able to state our result. For a diagonal section $\delta$ with $\delta(t)<t$ for all $t\in(0,1)$, define for $u,v\in[0,1]$, $u\leq v$,

[TABLE]

and for $u\geq v$, set $\bar{c}_{\delta}(u,v)=\bar{c}_{\delta}(v,u)$. Butucea et al. [16, Proposition 2.2] show that $\bar{c}_{\delta}$ is the density of a symmetric copula with diagonal section $\delta$. Note that the derivative $\delta^{\prime}$ of $\delta$ exists almost everywhere as $\delta$ is Lipschitz continuous. For a general diagonal section $\delta$, due to its continuity, the set $\{t\in[0,1]:\delta(t)<t\}$ is the union of disjoint open intervals $(\alpha_{j},\beta_{j})$, $j\in J$ for an at most countable index set $J$. Note that $\delta(\alpha_{j})=\alpha_{j}$ and $\delta(\beta_{j})=\beta_{j}$. For $u,v\in[0,1]$, define

[TABLE]

where $\bar{c}$ is given at (22), and

[TABLE]

Based on the results of [16], we arrive at the following theorem. Recall that $H=F_{2}-F_{1}$.

Theorem 11**.**

Let $(X_{1},X_{2})$ be a random vector with marginals $X_{1}$ and $X_{2}$ satisfying $\mathbb{P}(X_{1}\geq X_{2})=1$. Suppose that $X_{1}$ and $X_{2}$ have identical support being a (possibly unbounded) interval $[x^{L},x^{U}]$, and that their cdfs $F_{1}$ and $F_{2}$ are absolutely continuous with densities $f_{1}$ and $f_{2}$. If

[TABLE]

then

[TABLE]

where the supremum is taken over all possible joint laws of $(X_{1},X_{2})$. The maximum is attained when the joint density of $(X_{1},X_{2})$ is given by

[TABLE]

and $c_{D}$ is defined at (23). If (24) does not hold, then $\sup_{F}\operatorname{ent}(F)=-\infty$.

Proof*.*

By substitution,

[TABLE]

The result now follows from [16, Th. 2.5] in combination with (21) and Theorem 2. ∎

The condition $D(t)<t$ for all $t\in(0,1)$ is equivalent to $F_{2}(z)>F_{1}(z)$ for all $z\in(x^{L},x^{U})$. If this condition holds, then $c_{D}=\bar{c}_{D}$ and the formula for the entropy maximizing $f$ in Theorem 11 simplifies to

[TABLE]

for $x_{1},x_{2}\in[x^{L},x^{U}]$, $x_{1}\geq x_{2}$.

6 Examples

Example 7**.**

Let $X_{2}$ be uniformly distributed on $[0,1]$, and let $X_{1}$ be distributed as the maximum of $X_{2}$ and another independent uniformly distributed random variable, that is, $F_{1}(x)=x^{2}$. In this case $H(s)=s-s^{2}$ is unimodal with the maximum at $r=1/2$, and $t(s)=1-s$ for $s\in(r,1]$. The top-left panel of Fig. 1 shows a sample from the distribution $L$. It is easily seen that these values belong to the set $A$ given by (8) which consists here of the diagonal of the square $[0,1]^{2}$ and the lower part of the off-diagonal.

The smallest values for Kendall’s tau and Spearman’s rho are $1/3$ and $1/4$, respectively. The joint density with the maximal entropy is given by

[TABLE]

A sample from this distribution is shown in the bottom panel of Fig. 1. Note that in this example it is easy to simulate from the distribution $L$, and also from the distribution with density $f$ given at (25). To simulate a random vector $(X_{1},X_{2})$ with distribution $L$, generate a random variable $U$ which is uniformly distributed on $[0,1]$ and set

[TABLE]

A random vector $(X_{1},X_{2})$ with distribution given by the density $f$ at (25) is obtained by simulating independent random variables $U,V$ both uniformly distributed on $[0,1]$ and defining

[TABLE]

Example 8**.**

Theorem 1 establishes a relationship between the distribution of $(X_{1},X_{2})$ and the order statistics of a suitably chosen exchangeable pair $(V_{1},V_{2})$. Assume that $V_{1}$ and $V_{2}$ are independent. Then $\mathbb{P}(V_{1}\leq x,V_{2}\leq x)=G(x)^{2}$. By Theorem 1, $F_{1}(x)=G(x)^{2}$, whence

[TABLE]

Then

[TABLE]

is always unimodal. If $F_{1}$ is continuous the maximum attained at any lower quartile of $X_{1}$. The smallest possible Kendall’s tau equals $-1/3$; it does not depend on $F_{1}$. If we assume additionally that the support of $F_{1}$ is an interval, Theorem 10 applies and the equation used to find $t(s)$ turns into $\sqrt{F_{1}(t(s))}=1-\sqrt{F_{1}(s)}$. Substituting this in (19) yields that Spearman’s rho equals $-1/2$ for all $F_{1}$. If $F_{1}$ is absolutely continuous with density $f_{1}$ the maximum entropy is attained on the density $f(x_{1},x_{2})=2g(x_{1})g(x_{2})$, $x_{1}\geq x_{2}$, where $g(x)=f_{1}(x)/(2\sqrt{F_{1}(x)})$.

Example 9**.**

Let $X_{1}$ be uniform on $[0,1]$, and let $X_{2}=X_{1}^{1/\alpha}$ with $\alpha\in(0,1]$. Then $F_{1}(x)=x$, $F_{2}(x)=x^{\alpha}$, and

[TABLE]

for $x_{1}\geq x_{2}$, see (2). In this case, (9) yields that

[TABLE]

The function $F_{2}-F_{1}$ is unimodal and attains its maximum at $r=\alpha^{(1-\alpha)^{-1}}$. The smallest Kendall’s tau is

[TABLE]

Note that $\tau=1$ if $\alpha=1$, $\tau=0$ if $\alpha=1/3$, and $\tau\to-1$ as $\alpha\downarrow 0$.

We are not able to provide an explicit formula for Spearman’s rho in terms of $\alpha$ but Fig. 2 shows $\tau$ and $\rho$ as a function of $\alpha$.

Example 10** (Unimodal densities).**

Let $Z$ be a random variable with cdf $F$ and unimodal density $f$ whose support is $\mathbb{R}$. Let $X_{i}=Z+\mu_{i}$, $i=1,2$, with $\mu_{1}\geq\mu_{2}$, hence $F_{i}=F(\cdot-\mu_{i})$, $i\in\{1,2\}$. Then Theorem 9 yields for Kendall’s tau of the distribution $L$ at (9)

[TABLE]

which is the convolution of $\check{F}(z):=1-F(-z)$ and $F$.

Let us additionally assume that $f$ is symmetric about its mode at zero. Then $\check{F}=F$, and the unimodal function $F_{2}-F_{1}$ has its maximum at $r=(\mu_{1}+\mu_{2})/2$. Hence, the function $t:(r,\infty)\to(-\infty,r)$ in Theorem 10 is given by $t(s)=\mu_{2}+\mu_{1}-s$. If $F=\Phi$ is the standard Gaussian cdf, then $\Phi*\Phi(x)=\Phi(x/\sqrt{2})$, and, therefore,

[TABLE]

In particular, $\tau=1$ if $\mu_{1}=\mu_{2}$, $\tau=0$ if $\mu_{1}-\mu_{2}=\sqrt{2}\Phi^{-1}(3/4)\approx 0.954$, and if $\mu_{1}-\mu_{2}\to\infty$, then $\tau\to-1$.

For Spearman’s rho, we numerically computed the integrals in (19) for $F=\Phi$. The values of $\tau$ and $\rho$ as functions of $\mu_{1}-\mu_{2}\geq 0$ are displayed in Fig. 3.

Example 11** (Exponential marginal distributions).**

Let $Z$ be an exponential random variable with cdf $F(x)=1-e^{-x}$, and let $X_{i}=Z/\lambda_{i}$, $i\in\{1,2\}$, with $\lambda_{1}\leq\lambda_{2}$. Considering $\log(X_{i})$, $i\in\{1,2\}$ shows that we are in the same setting as in Example 10 because Kendall’s tau and Spearman’s rho are invariant under monotone transformations of the marginals and the stochastic ordering is preserved if we transform both marginals with the same increasing function.

However, we can also compute $\tau$ and $\rho$ directly. Kendall’s tau is given by

[TABLE]

Note that $\tau=1$ if $\lambda_{1}=\lambda_{2}$, $\tau=0$ if $\lambda_{1}/\lambda_{2}=1/3$, and $\tau\to-1$ as $\lambda_{1}/\lambda_{2}\downarrow 0$.

The function

[TABLE]

is unimodal on $[0,\infty)$ with maximum at

[TABLE]

Therefore, we can use (19) to compute $\rho$. Considering the increasing transformation $\lambda_{1}X_{i}$, $i\in\{1,2\}$, we see that $\tau$ and $\rho$ only depend on $\lambda_{1}/\lambda_{2}\leq 1$. Fig. 4 provides plots of $\tau$ and $\rho$ as functions of $\lambda_{1}/\lambda_{2}\in(0,1]$.

7 Generalizations

A multivariate version of Theorem 1 is the following.

Theorem 12**.**

A random vector $(X_{1},\dots,X_{n})$ with marginal cdfs $F_{1},\dots,F_{n}$ satisfies $\mathbb{P}(X_{1}\geq\dots\geq X_{n})=1$ if and only if $X_{i}=V_{(i)}$ where $V_{(1)}\geq\dots\geq V_{(n)}$ are the order statistics of a random vector $(V_{1},\dots,V_{n})$ with exchangeable components and such that for $j=1,\dots,n$

[TABLE]

Proof*.*

Let $(V_{1},\dots,V_{n})$ be an exchangeable random vector that satisfies the above condition. By [17, Proposition 4.4.1], we have that

[TABLE]

Conversely, if the vector $(V_{1},\dots,V_{n})$ is obtained as the random permutation of $(X_{1},\dots,X_{n})$, then it is exchangeable and the formula for $\mathbb{P}(V_{1}\leq x,\dots,V_{j}\leq x)$ is essentially the inversion of the first equality in the above equation. ∎

A variant of Theorem 1 applies to random elements in a lattice $E$ with partial order $\preceq$, and with $\vee$ being the maximum and $\wedge$ being the minimum operation. Endow $E$ with the $\sigma$-algebra generated by $\{y:\;y\preceq x\}$ for all $x\in E$. Since these events form a $\pi$-system, the values $\mathbb{P}(X\preceq x)$, $x\in E$, uniquely determine the distribution of an $E$-valued random element $X$.

In this case, Theorem 1 admits a direct generalization. Namely $X_{1}\preceq X_{2}$ a.s. if and only if $X_{1}=V_{1}\vee V_{2}$ and $X_{2}=V_{1}\wedge V_{2}$ for a pair $(V_{1},V_{2})$ of exchangeable random elements in $E$ such that

[TABLE]

and $\mathbb{P}(V_{1}\preceq x,V_{2}\preceq x)=\mathbb{P}(X_{1}\preceq x)$.

Acknowledgement

The problem of characterizing bivariate copulas with stochastically ordered marginals was brought to the attention of the second author by Nicholas Kiefer from the Economics Department at Cornell University.

The authors are grateful to the referees for spotting mistakes in the original version of this paper and suggesting numerous improvements.