Uniform spaces and the Newtonian structure of (big)data affinity kernels

TL;DR

This paper demonstrates that under mild conditions, data affinity kernels can be represented as Newtonian potentials with a quasi-metric structure, revealing an underlying geometric framework for big data analysis.

Contribution

It establishes that affinity kernels satisfying specific conditions can be expressed as Newtonian potentials with an associated quasi-metric, linking data affinity to geometric structures.

Findings

Affinity kernels can be represented as Newtonian potentials.

The underlying space admits a quasi-metric structure.

A quantitative transitivity condition is established.

Abstract

Let $X$ be a (data) set. Let $K(x,y)>0$ be a measure of the affinity between the data points $x$ and $y$. We prove that $K$ has the structure of a Newtonian potential $K(x,y)=\varphi(d(x,y))$ with $\varphi$ decreasing and $d$ a quasi-metric on $X$ under two mild conditions on $K$. The first is that the affinity of each $x$ to itself is infinite and that for $x\neq y$ the affinity is positive and finite. The second is a quantitative transitivity; if the affinity between $x$ and $y$ is larger than $\lambda>0$ and the affinity of $y$ and $z$ is also larger than $\lambda$, then the affinity between $x$ and $z$ is larger than $\nu(\lambda)$. The function $\nu$ is concave, increasing, continuous from $\mathbb{R}^+$ onto $\mathbb{R}^+$ with $\nu(\lambda)<\lambda$ for every $\lambda>0$.

Equations

$$K(x,y)=\phi (d(x,y)).$$

$$a_{1}d(x,y)\le {\rho }^{\beta }(x,y)\le a_{2}d(x,y)$$

$${\mathcal{V}}_{\delta }(r)\subseteq \mathcal{V}(r)\subseteq {\mathcal{V}}_{\delta }((1+\gamma )r)$$

$$\frac{1}{4^{\beta }}\delta (x,y)\le \rho (x,y{)}^{\beta }\le 2^{\beta }\delta (x,y),$$

$$\psi (\nu (\lambda ))\le M\psi (\lambda )$$

$$\frac{M^{k+1}-M^k}{{\lambda }_k-{\lambda }_{k+1}}=\frac{M^{k+1}-\psi (\lambda )}{\lambda -{\lambda }_{k+1}}.$$

$$\frac{M^{k+2}-M^{k+1}}{{\lambda }_{k+1}-{\lambda }_{k+2}}=\frac{M^{k+2}-\psi (\nu (\lambda ))}{\nu (\lambda )-{\lambda }_{k+2}}.$$

$$\psi (\lambda )=M^{k+1}-M^k(M-1)\frac{\lambda -{\lambda }_{k+1}}{{\lambda }_k-{\lambda }_{k+1}}$$

$$\psi (\nu (\lambda ))=M\left(M^{k+1}-M^k(M-1)\frac{\nu (\lambda )-{\lambda }_{k+2}}{{\lambda }_{k+1}-{\lambda }_{k+2}}\right).$$

$$\frac{\nu (\lambda )-\nu ({\lambda }_{k+1})}{\lambda -{\lambda }_{k+1}}\ge \frac{\nu ({\lambda }_k)-\nu ({\lambda }_{k+1})}{{\lambda }_k-{\lambda }_{k+1}},$$

$$K(x,y)=\phi (h(x,y)\rho (x,y)).$$

$$\mathcal{V}(r)=E_{\eta (r)}=\{(x,y):K(x,y)>\eta (r)\}.$$

$$\left\{\rho <\frac{s}{4}\right\}\subseteq \left\{(\psi \circ K{)}^{1/\beta }>s\right\}\subseteq \left\{\rho <2^{1+1/\beta }s\right\}$$

$$\frac{1}{4}(\psi \circ K{)}^{1/\beta }\le \rho \le 2^{1/\beta }(\psi \circ K{)}^{1/\beta }.$$

Full text

1. Introduction

Isaac Newton in the seventeenth century started the endless quantitative approach to the understanding of nature. The quantitative character of the formulation of the Law of Universal Gravitation, should not hide its deep qualitative aspects. Now, more than 300 years later, we are able to explain the central fields as gradients of radial potentials centered at the “object” generating the field. Usually the potential take the form $V(x)=\varphi(\left|x\right|)$ where $\varphi$ is a decreasing profile and $\left|x\right|$ is the distance of the point $x$, where the field is to be measured, to the origin of coordinates supporting the mass or the charge that generates the field. In the Euclidean $n$-dimensional space the profile $\varphi(r)=r^{-n+2}$ gives the fundamental solution of the Laplacian. And the kernel $K(x,y)=\varphi(\left|x-y\right|)$ provides the basic information in order to produce the continuous models by convolution with the mass or charge densities that determine the system. These facts are also the starting points for harmonic analysis.

Our aim in this paper is to use arguments and results strongly related to the theory of uniform spaces, in order to give sufficient conditions on a kernel function $K(x,y)$ defined on an abstract setting, in such a way that $K(x,y)=\varphi(d(x,y))$ with $\varphi$ a decreasing profile and $d$ a (quasi)metric on $X$.

In other words, we aim to obtain an abstract form of Newtonian potentials for general kernels. Our approach in the search of conditions on the kernel $K(x,y)$ will be based in the heuristic associated to the interpretation of $K(x,y)$ as an affinity matrix for (big) data. Amazingly enough the abstract of the paper [2] of the Coifman Group leading the harmonic analysis approach to determine the structure of big data sets, reads “The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic by integration.” In some sense the result of this paper shows that also the potential theoretic Newtonian view of nature is still close to these problems. By the way, our approach could be a good example of how qualitative aspects of a system leads to structural results of the model.

Let $X$ be a set (data set). Each element $x$ of $X$ is understood as a data point. Let $K(x,y)$ be a nonnegative number measuring the affinity between the two data points $x$ and $y$. We shall consider some basic properties of affinity which will be sufficient to obtain the Newtonian potential type structure for $K$. Symmetry; affinity is a symmetric relation on $X\times X$ ($K(x,y)=K(y,x)$ for every $x$, $y\in X$). Positivity; there is positive affinity between any couple of data points $x$ and $y$ ($K(x,y)\geq 0$ for every $x$, $y\in X$). Diagonal singularity; the self affinity is unimprovable. Precisely, the affinity of each data point $x$ with itself is $+\infty$ ($K(x,x)=+\infty$ for every $x$) but for $x\neq y$ the affinity is finite ($K(x,y)<\infty$ for $x\neq y$). Quantitative Transitivity; if the affinity between the data points $x$ and $y$ is larger than $\lambda>0$ and the affinity between $y$ and $z$ is larger than $\lambda$ then the affinity between the points $x$ and $z$ is larger than $\nu(\lambda)<\lambda$. Here $\nu$ is a nonnegative, concave, increasing and continuous function defined on $\mathbb{R}^{+}$ onto $\mathbb{R}^{+}$ such that $\nu(\lambda)<\lambda$.

A quasi-metric in $X$ is a nonnegative symmetric function $d$ defined on $X\times X$ which vanishes only on the diagonal $\Delta$ of $X\times X$ and satisfies a weak form of the triangle inequality, there exists a constant $\tau$ ($\geq 1$) such that the inequality $d(x,z)\leq\tau(d(x,y)+d(y,z))$ holds for every $x$, $y$ and $z$ in $X$.

The main result of this paper can be stated as follows.

Theorem 1**.**

Let $X$ be a set. Let $K:X\times X\to\mathbb{R}^{+}$ be a symmetric function satisfying the singularity and the quantitative transitivity conditions. Then there exist a decreasing and continuous function $\varphi$ defined in $\mathbb{R}^{+}$ and a quasi-metric $d$ on $X$ such that

[TABLE]

Moreover, $d(x,y)=h(x,y)\rho(x,y)$ with $\rho$ a metric on $X$ and $h$ a symmetric function such that for some constants $0<c_{1}<c_{2}<\infty$ satisfies $c_{1}\leq h(x,y)\leq c_{2}$ for every $x$ and $y$ in $X$.

The deepest results on the structure of quasi-metrics are due to Macías and Segovia and are contained in [5]. See also [1]. The most significant for our purposes is the fact that each quasi-metric is equivalent to a power of a metric. In other words, given a quasi-metric $d$ on $X$ with constant $\tau$, there exist $\beta\geq 1$ and a metric $\rho$ on $X$ such that for some positive constants $a_{1}$ and $a_{2}$ the inequalities

[TABLE]

hold for every $x$ and $y$ in $X$. Actually the proof is based in Frink’s lemma of metrization of uniform structures with a countable basis ([3], [4]).

The rest of the paper is organized in the following way. Section 2 gives a characterization of quasi-metrics on a set in terms of properties of the family of stripes in $X\times X$ induced by the quasi-metric. Section 3 contains the construction of the monotonic profile. In Section 4 we prove the main result.

2. Quasi-metrics and families of stripes around the diagonal

Let $X$ be a set. The composition of two subsets $U$ and $V$ of $X\times X$ is given by $V\circ U=\{(x,z)\in X\times X:\textrm{ there exists }y\in X\textrm{ such that }(x,y)\in U\textrm{ and }(y,z)\in V\}$. A subset $U$ in $X\times X$ is said to be symmetric if $(x,y)\in U$ if and only if $(y,x)\in U$. Set $\Delta$ to denote the diagonal in $X\times X$, i.e. $\Delta=\left\{(x,x):x\in X\right\}$. When a quasi-metric $\delta$ with constant $\tau$ is given in $X$, it is easy to check that the one parameter family $\mathcal{V}(r)=\{(x,y)\in X\times X:\delta(x,y)<r\}$; $r>0$, of stripes around the diagonal of $X\times X$, satisfies

1. (S1)

each $\mathcal{V}(r)$ is symmetric; 2. (S2)

$\Delta\subseteq\mathcal{V}(r)$, for every $r>0$; 3. (S3)

$\mathcal{V}(r_{1})\subseteq\mathcal{V}(r_{2})$, for $0<r_{1}\leq r_{2}$; 4. (S4)

$\cup_{r>0}\mathcal{V}(r)=X\times X$; 5. (S5)

$\cap_{r>0}\mathcal{V}(r)\subseteq\Delta$; 6. (S6)

there exists $T\geq 1$ such that $\mathcal{V}(r)\circ\mathcal{V}(r)\subseteq\mathcal{V}(Tr)$, for every $r>0$.

Actually, the constant $T$ in (S6) can be taken to be the triangle constant $\tau$ of $\delta$. Set $\mathcal{P}(X\times X)$ to denote the set of subsets of $X\times X$.

Theorem 2**.**

Let $\mathcal{V}:\mathbb{R}^{+}\to\mathcal{P}(X\times X)$ be a one parameter family of the subsets of $X\times X$ that satisfies (S1) to (S6) above. Then the function $\delta$ defined on $X\times X$ by $\delta(x,y)=\inf\{r>0:(x,y)\in\mathcal{V}(r)\}$ is a quasi-metric on $X$ with $\tau\leq T$. Moreover, for each $\gamma>0$, we have

[TABLE]

hold for every $r>0$, where $\mathcal{V}_{\delta}(s)=\{(x,y)\in X\times X:\delta(x,y)<s\}$.

Proof.

From (S4) for the family $\mathcal{V}$ we see that $\delta(x,y)$ is well defined as a nonnegative real number. The symmetry of $\delta$ follows from (S1). The fact that $\delta$ vanishes on the diagonal $\Delta$ follows from $\Delta\subseteq\cap_{r>0}\mathcal{V}(r)$ which is contained in (S2). Now, if $(x,y)\in X\times X$ and $\delta(x,y)=0$, then, from (S3) for each $r>0$, $(x,y)\in\mathcal{V}(r)$. Now, from (S5) we necessarily have that $(x,y)\in\Delta$ or, in other words $x=y$. Let us check that $\delta$ satisfies a triangle inequality. Let $x$, $y$ and $z$ be three points in $X$. Let $\varepsilon>0$. Take $r_{1}>0$ and $r_{2}>0$ such that $r_{1}<\delta(x,y)+\varepsilon$, $r_{2}<\delta(y,z)+\varepsilon$, $(x,y)\in\mathcal{V}(r_{1})$ and $(y,z)\in\mathcal{V}(r_{2})$. From (S6) with $r^{*}=\sup\{r_{1},r_{2}\}$, we have $(x,z)\in\mathcal{V}(r_{2})\circ\mathcal{V}(r_{1})\subseteq\mathcal{V}(r^{*})\circ\mathcal{V}(r^{*})\subseteq\mathcal{V}(Tr^{*})$. Hence $\delta(x,z)\leq Tr^{*}\leq T(r_{1}+r_{2})\leq T(\delta(x,y)+\delta(y,z))+2\varepsilon T$ and we get the triangle inequality with $\tau=T$. Notice first that from (S3), $\mathcal{V}_{\delta}(r)\subseteq\mathcal{V}(r)$ for every $r>0$. Take now $(x,y)\in\mathcal{V}(r)$, then $\delta(x,y)\leq r<(1+\gamma)r$, so that $\mathcal{V}(r)\subseteq\mathcal{V}_{\delta}((1+\gamma)r)$ for every $\gamma>0$ and every $r>0$. ∎

The next result follows from the above and the metrization theorem of quasi-metric spaces proved in [5] as an application of Frink’s Lemma on metrizability of uniform spaces with countable bases.

Theorem 3**.**

Let $\mathcal{V}$ and $\delta$ be as in Theorem 2. Then, there exist a constant $\beta\geq 1$ and a metric $\rho$ on $X$ such that

1. (i)

$4^{-\beta}\delta\leq\rho^{\beta}\leq 2^{\beta}\delta$; 2. (ii)

$\mathcal{V}_{\rho^{\beta}}\left(\tfrac{r}{4^{\beta}}\right)\subseteq\mathcal{V}(r)\subseteq\mathcal{V}_{\rho^{\beta}}(2^{\beta+1}r)$* where $\mathcal{V}_{\rho^{\beta}}(r)=\{(x,y)\in X\times X:\rho^{\beta}(x,y)<r\}$.*

Proof.

Following the proof of Theorem 2 on page 261 in [5] take $\alpha<1$ such that $(3T^{2})^{\alpha}=2$ and $\beta=\tfrac{1}{\alpha}>1$. With $\rho$ the metric provided by the metrization theorem for uniform spaces with countable bases, we have that $4\rho(x,y)\geq\delta(x,y)^{\alpha}\geq\tfrac{1}{2}\rho(x,y)$. So that

[TABLE]

and (ii) follow from these inequalities and (2.1) with $\gamma=1$. ∎

3. Building the basic profile shape

The classical inverse proportionality to the square of the distance between the two bodies for the gravitation field, translates into inverse proportionality to the distance for the potential. That is $\varphi(r)=\tfrac{1}{r}$ for the gravitational potential.

This section is devoted to the construction of the basic shapes of the profiles that allow the Newtonian representation of the kernels. This construction requires to solve a functional inequality involving the function $\nu$ that controls the quantitative transitive property of $K$.

Proposition 4**.**

Let $\nu$ be a concave, continuous, nonnegative and increasing function defined on $\mathbb{R}^{+}$ onto $\mathbb{R}^{+}$ such that $\nu(\lambda)<\lambda$ for every $\lambda>0$. Then, given $M>1$, there exists a continuous, decreasing and convex function $\psi$ defined on $\mathbb{R}^{+}$ with $\psi(1)=1$ such that the inequality

[TABLE]

holds for every $\lambda>0$.

Proof.

Set $\lambda_{0}=1$, $\lambda_{1}=\nu(1)$ and $\lambda_{-1}=\nu^{-1}(1)$. Notice that $\lambda_{1}<1$ and $\lambda_{-1}>1$. In fact, $\lambda_{1}=\nu(1)<1$ and $1=\nu^{-1}(\nu(1))=\nu^{-1}(\lambda_{1})<\nu^{-1}(1)=\lambda_{-1}$. Set for $k\in\mathbb{N}$, $\lambda_{k}=\nu(\lambda_{k-1})$ and $\lambda_{-k}=\nu^{-1}(\lambda_{-k+1})$. Notice that $\lambda_{k}$ decreases as $k\to\infty$ and increases when $k\to-\infty$. The continuity of $\nu$ and the property $\nu(\lambda)<\lambda$ for every positive $\lambda$ imply that $\lambda_{k}\to 0$ as $k\to\infty$ and $\lambda_{k}\to\infty$ as $k\to-\infty$ monotonically. This basic sequence $\{\lambda_{k}:k\in\mathbb{Z}\}$ allows to construct a function $\psi$ in the following way. Set $\psi(\lambda_{k})=M^{k}$, $k\in\mathbb{Z}$ and for $\lambda\in[\lambda_{k+1},\lambda_{k}]$ define $\psi(\lambda)$ by linear interpolation. It is clear that $\psi$ is continuous, decreasing, $\psi(0^{+})=+\infty$, $\psi(\infty)=0$, $\psi(1)=\psi(\lambda_{0})=M^{0}=1$ and that $\psi$ is convex. We only have to check that $\psi$ solves inequality (3.1). On the sequence $\{\lambda_{k}:k\in\mathbb{Z}\}$, (3.1) becomes an equality. In fact, $\psi(\nu(\lambda_{k}))=\psi(\lambda_{k+1})=M^{k+1}=MM^{k}=M\psi(\lambda_{k})$.

Let us now take $\lambda\in(\lambda_{k+1},\lambda_{k})$ for $k\in\mathbb{Z}$. For such $\lambda$, $\psi(\lambda)$ satisfies

[TABLE]

On the other hand, since $\lambda_{k+1}<\lambda<\lambda_{k}$, we have that $\lambda_{k+2}=\nu(\lambda_{k+1})<\nu(\lambda)<\nu(\lambda_{k})=\lambda_{k+1}$. Hence $\psi(\nu(\lambda))$ satisfies

[TABLE]

From (3.2) and (3.3) we get

[TABLE]

and

[TABLE]

Now, since $\nu$ is concave, we have for $\lambda_{k+1}<\lambda<\lambda_{k}$ that

[TABLE]

hence, $\psi(\nu(\lambda))\leq M\psi(\lambda)$. ∎

The basic shapes for the profiles $\varphi$ in our main result will be given as composition of the inverse $\eta$ of $\psi$ with power laws.

4. Proof of the main result

Let us start by rewriting, formally, the properties of symmetry, positivity, singularity and transitivity of a data affinity kernel $K(x,y)$ defined on the set $X\times X$. Let $K:X\times X\to\mathbb{R}$ such that

1. (K1)

$K(x,y)=K(y,x)$, for every $x$ and $y$ in $X$; 2. (K2)

$K(x,y)>0$, for every $x$ and $y$ in $X$; 3. (K3)

$K(x,y)=+\infty$ if and only if $x=y$; 4. (K4)

there exists a continuous, concave, increasing and nonnegative function $\nu$ defined on $\mathbb{R}^{+}$ onto $\mathbb{R}^{+}$, with $\nu(\lambda)<\lambda$, $\lambda>0$, such that $K(x,z)>\nu(\lambda)$ whenever there exists $y\in X$ with $K(x,y)>\lambda$ and $K(y,z)>\lambda$, for every $\lambda>0$.

With these properties, Theorem 1 can be restated as follows.

Theorem 5**.**

Let $X$ be a set. Let $K$ be a kernel on $X\times X$ satisfying properties(K1) to (K4). Then, there exist a metric $\rho$ on $X$, a real number $\beta\geq 1$, a function $h(x,y)$ defined on $X\times X$ with $2^{-1/\beta}\leq h(x,y)\leq 4$ and a function $\varphi:\mathbb{R}^{+}\to\mathbb{R}^{+}$ continuous, decreasing with $\varphi(0^{+})=+\infty$ and $\varphi(\infty)=0$, such that

[TABLE]

Proof.

Let $\nu$ be the function provided by (K4). Let $\psi$ be given by Proposition 4 with this function $\nu$, and $M=2$. Hence $\psi(\nu(\lambda))\leq 2\psi(\lambda)$ for every $\lambda>0$. Take $\eta=\psi^{-1}$ and $\mathcal{V}:\mathbb{R}^{+}\to\mathcal{P}(X\times X)$ given by

[TABLE]

Let is check that $\mathcal{V}$ satisfies properties (S1) to (S6) in Section 2 with constant $T=2$ ($=M$). From (K1) we see that each $E_{\lambda}$ is symmetric, in particular $\mathcal{V}(r)$ is symmetric for every $r>0$. Since $K(x,x)=+\infty$, from (K3), we have that $\Delta\subseteq E_{\eta(r)}=\mathcal{V}(r)$ for $r>0$. In order to check (S3) take $0<r_{1}<r_{2}<\infty$. Hence $\eta(r_{1})>\eta(r_{2})$, so that $K(x,y)>\eta(r_{1})$ implies $K(x,y)>\eta(r_{2})$. Or, in other words $E_{\eta(r_{1})}\subset E_{\eta(r_{2})}$. Or $\mathcal{V}(r_{1})\subseteq\mathcal{V}(r_{2})$. The positivity (K2) of $K(x,y)$ shows (S4). Property (S5) of $\mathcal{V}$ follows from (S3). To prove (S6) for $\mathcal{V}$, take $r>0$. If $(x,z)\in\mathcal{V}(r)\circ\mathcal{V}(r)=E_{\eta(r)}\circ E_{\eta(r)}$, then there exists $y\in X$ such that $K(x,y)>\eta(r)$ and $K(y,z)>\eta(r)$. From (K4), $K(x,z)>\nu(\eta(r))$. Now applying (3.1) with $\lambda=\psi^{-1}(r)$ we get $K(x,z)>\nu(\psi^{-1}(r))\geq\eta(2r)$, or $(x,z)\in E_{\eta(2r)}=\mathcal{V}(2r)$. Hence (S6) for $\mathcal{V}$ holds with $T=2$. We can, then apply the results of Section 2. First to produce a quasi-metric $\delta$ as in Theorem 2 and then to provide the metric $\rho$ and the exponent $\beta$ given in Theorem 3. Thus, for every $r>0$, $\mathcal{V}_{\rho^{\beta}}\left(\tfrac{r}{4^{\beta}}\right)\subseteq\mathcal{V}(r)=E_{\eta(r)}\subseteq\mathcal{V}_{\rho^{\beta}}(2^{\beta+1}r)$, where $\rho$ is a metric in $X$ and, since $T$ can be taken to be equal to 2, $\beta=\log_{2}12$ works. The above inclusiones, taking $s=r^{1/\beta}$, are equivalent to

[TABLE]

for every $s>0$. Let $x$ and $y$ be two different points in $X$. Since $0<K(x,y)<\infty$ so is $(\psi\circ K)^{1/\beta}(x,y)$. There exists, then, a unique $j\in\mathbb{Z}$ ($j=j(x,y)$) such that $2^{j}\leq(\psi\circ K)^{1/\beta}(x,y)<2^{j+1}$. By the second inclusion in (4.1) we see that $\rho(x,y)<2^{1/\beta}2^{j}\leq 2^{1/\beta}(\psi\circ K)^{1/\beta}(x,y)$. On the other hand, since $(\psi\circ K)^{1/\beta}(x,y)<2^{j+1}$, from the first inclusion in (4.1) we necessarily have that $\rho(x,y)\geq\tfrac{2^{j+1}}{4}>\tfrac{1}{4}(\psi\circ K)^{1/\beta}(x,y)$. Hence for $x\neq y$ we have

[TABLE]

Set $h(x,y)=\tfrac{(\psi\circ K)^{1/\beta}}{\rho(x,y)}$ for $x\neq y$ and $h(x,x)=1$. Then $\tfrac{1}{2^{1/\beta}}\leq h\leq 4$ and $K(x,y)=\psi^{-1}((h(x,y)\rho(x,y))^{\beta})=\varphi(h(x,y)\rho(x,y))$ with $\varphi(r)=\psi^{-1}(r^{\beta})=\eta(r^{\beta})$. ∎

Notice that since $h$ is symmetric and bounded above and below by positive constants the function $d(x,y)=h(x,y)\rho(x,y)$ is a quasi-metric. But actually $d$ is better than a general quasi-metric since its triangular inequality constant can be taken to be independent of the length of chains. Precisely, $d(x_{1},x_{m})\leq 2^{2+1/\beta}\sum_{j=1}^{m-1}d(x_{j},x_{j+1})$.

Let us observe also that Newtonian type power laws are obtained when $\nu(\lambda)=a\lambda$ for $a<1$. In fact, with $m=\tfrac{1}{\log_{2}a}<0$, $\psi(r)=r^{m}$ solves the equation $(a\lambda)^{m}=2\lambda^{m}$. Hence $\varphi$ becomes also a power law.