Making the Dynamic Time Warping Distance Warping-Invariant

TL;DR

This paper introduces a new warping-invariant semi-metric called the twi-distance, addressing the non-invariance of DTW, leading to more efficient and consistent time series classification.

Contribution

The paper proposes the twi-distance, a warping-invariant semi-metric, improving consistency and efficiency over the traditional DTW distance in time series analysis.

Findings

Twi-distance is practically equivalent to DTW in classification error rates.

Twi-distance requires less storage and computation time than DTW.

The results challenge the common use of DTW in nearest-neighbor classification.

Abstract

The literature postulates that the dynamic time warping (dtw) distance can cope with temporal variations but stores and processes time series in a form as if the dtw-distance cannot cope with such variations. To address this inconsistency, we first show that the dtw-distance is not warping-invariant. The lack of warping-invariance contributes to the inconsistency mentioned above and to a strange behavior. To eliminate these peculiarities, we convert the dtw-distance to a warping-invariant semi-metric, called time-warp-invariant (twi) distance. Empirical results suggest that the error rates of the twi and dtw nearest-neighbor classifier are practically equivalent in a Bayesian sense. However, the twi-distance requires less storage and computation time than the dtw-distance for a broad range of problems. These results challenge the current practice of applying the dtw-distance in nearest-neighbor classification and suggest the proposed twi-distance as a more efficient and consistent option.

Figures

Tables

Table 1. Table 1: Percentage of datasets for which row-classifier was practically better (left) / worse (right) than column-classifier.

	row $\gg$ column						row $\ll$ column
	euc	dtw	bdtw${}_{\text{lb}}$	bdtw${}_{\text{ub}}$	twi		euc	dtw	bdtw${}_{\text{lb}}$	bdtw${}_{\text{ub}}$	twi
euc	0.0	36.1	32.5	34.9	32.5	euc	0.0	48.2	48.2	47.0	48.2
dtw	48.2	0.0	7.2	8.4	8.4	dtw	36.1	0.0	4.8	7.2	10.8
bdtw${}_{\text{lb}}$	48.2	4.8	0.0	7.2	1.2	bdtw${}_{\text{lb}}$	32.5	7.2	0.0	6.0	3.6
bdtw${}_{\text{ub}}$	47.0	7.2	6.0	0.0	6.0	bdtw${}_{\text{ub}}$	34.9	8.4	7.2	0.0	7.2
twi	48.2	10.8	3.6	7.2	0.0	twi	32.5	8.4	1.2	6.0	0.0

Table 2. Table 2 : Characteristic properties of the AMPds2 dataset for different numbers d 𝑑 d of days.

	1d	7d	14d	30d
length of time series	$1,440$	$10,080$	$20,160$	$43,200$
train (test) size	6935	997	503	237
space-saving rate	73.8	73.8	73.8	73.8

Table 3. Table 3 : Error rates and computation time of seven nearest-neighbor classifiers on AMPds2 data.

	error rate					time (hh:mm:ss)
	1d	7d	14d	30d		1d	7d	14d	30d
euc	51.68	74.92	75.94	75.53		0:24:29	0:03:30	0:01:47	0:00:48
dtw	20.36	6.52	7.36	5.91		65:04:21	66:54:09	67:58:13	69:10:28
bdtw${}_{\text{lb}}$	20.53	6.12	6.76	5.91		11:58:07	11:48:49	11:51:05	11:49:37
bdtw${}_{\text{ub}}$	20.53	6.42	6.96	5.91		11:54:24	12:03:06	12:12:36	12:04:41
twi	20.69	6.72	7.55	5.91		6:15:32	6:32:18	6:51:55	6:52:44
opt-dtw	20.36	6.52	7.36	5.91		26:14:50	33:23:09	35:08:48	35:40:14
opt-twi	20.69	6.72	7.55	5.91		2:24:52	2:57:00	3:12:45	3:14:02

Table 4. Table 4 : Error rates of the three nn-classifiers on different variants of the cylinder-peak data. .

id	$b_{+1}$	$b_{-1}$	${\theta }_b$	$c_{+1}$	$c_{-1}$	${\theta }_c$	z-transform	euc-nn	dtw-nn	twi-nn
E1	0	$-$0.05	0.0	1.0	1.0	1.0	no	50 %	0 %	37 %
E2	0	$-$0.05	0.0	1.0	1.0	0.1	no	35 %	0 %	0 %
E3	0	0	1.0	1.0	1.05	1.0	no	50 %	31 %	0 %
E4	0	0	0.1	1.0	1.05	1.0	no	49 %	0 %	0 %
E5	0	$-$0.05	0.0	1.0	1.0	1.0	yes	51 %	45 %	45 %

Equations

$$[m]\times [n]=\left\{(i,j):i\in [m],j\in [n]\right\}.$$

$$C_p(x,y)={\Vert {\varphi }_p(x)-{\psi }_p(y)\Vert }^2=\sum _{(i,j)\in p}{\left(x_i-y_j\right)}^2,$$

$$\delta (x,y)=\min \left\{\sqrt{C_p(x,y)}:p\in {\mathcal{P}}_{m,n}\right\}.$$

$$x^{\prime }=(x_1,\dots ,\underset{r_1\text{-times}}{\underbrace{x_{i_1},\dots ,x_{i_1}}},x_{i_1+1},\dots ,\underset{r_2\text{-times}}{\underbrace{x_{i_2},\dots ,x_{i_2}}},x_{i_2+1},\dots ,\underset{r_k\text{-times}}{\underbrace{x_{i_k},\dots ,x_{i_k}}},x_{i_k+1}\dots ,x_n)$$

$$d(x,y)=d(x^{\prime },y^{\prime })$$

$$\delta (x,y{)}^2$$

$$\delta (x^{\prime },y{)}^2$$

$$[x]=\left\{y\in \mathcal{T}:y\sim x\right\}$$

$${\mathcal{T}}^{\ast }=\left\{[x]:x\in \mathcal{T}\right\}$$

$${\delta }^{\ast }:{\mathcal{T}}^{\ast }\times {\mathcal{T}}^{\ast }\to {\mathbb{R}}_{\ge 0},([x],[y])\mapsto \underset{x^{\prime }\in [x]}{\inf }\underset{y^{\prime }\in [y]}{\inf }\delta (x^{\prime },y^{\prime }).$$

$$[x]$$

$${\delta }^{\ast }([x],[y])=\delta (x^{\ast },y^{\ast })$$

$${\delta }^{\sim }:\mathcal{T}\times \mathcal{T}\to {\mathbb{R}}_{\ge 0},(x,y)\mapsto {\delta }^{\sim }(x,y)={\delta }^{\ast }([x],[y]).$$

$${\varphi }_1=\frac{\left|x\right|\left|y\right|}{\left|x^{\ast }\right|\left|y^{\ast }\right|+\left|x^{\ast }\right|+\left|y^{\ast }\right|}\text{and}{\varphi }_2={\rho }_{\text{c}}(x)\cdot {\rho }_{\text{c}}(y)=\frac{\left|x\right|\left|y\right|}{\left|x^{\ast }\right|\left|y^{\ast }\right|},$$

$${\varphi }_1\ge \frac{n^2}{m^2+2m}>\frac{n^2}{(m+1)}\ge 1$$

$$\delta (x,x^{\prime })$$

$$\delta (x,y)$$

$${\mathcal{V}}_{\mathcal{D}}(x)$$

$$F:\mathcal{T}\to \mathbb{R},z\mapsto \sum _{i=1}^n\delta {\left(x_i,z\right)}^2,$$

$$J:{\Pi }_k\to \mathbb{R},\left({\mathcal{C}}_1,\dots ,{\mathcal{C}}_k\right)\mapsto \sum _{i=1}^k\sum _{x\in {\mathcal{C}}_i}\delta (x,{\mu }_i{)}^2,$$

$$J\left({\mathcal{C}}_1,\dots ,{\mathcal{C}}_k\right)=\sum _{i=1}^k{F}_i({\mu }_i)$$

$$x_1$$

$$x_2$$

$${\mu }_2^r=(0.5,2,\underset{r-\text{times}}{\underbrace{3,\dots ,3}}),$$

$$F_{\text{cohesion}}({\mu }_1,{\mu }_2)=F_1({\mu }_1)+F_2({\mu }_2),$$

$$F_{\text{separation}}({\mu }_1,{\mu }_2)=\delta ({\mu }_1,{\mu }_2{)}^2.$$

$${\rho }_{\text{ss}}=1-\frac{\left|x^{\ast }\right|+\left|y^{\ast }\right|}{2(\left|x\right|+\left|y\right|)}.$$

$$\displaystyle x_{i}=\left\{\begin{array}[]{c@{\quad: \quad}l}b_{y}+\varepsilon_{b}&i\notin{\mathcal{I}}(x)\\ c_{y}+\varepsilon_{c}&i\in{\mathcal{I}}(x)\end{array}\right.,$$

$$z_{i}=\left\{\begin{array}[]{c@{\quad: \quad}l}x_{i}&1\leq i\leq m\\ y_{m-i}&m<i\leq m+n\end{array}\right.$$

$$x=x^{\prime }a=p_1\cdots p_{k}a\stackrel{(1)}{=}{p}_1\cdots p_{k-1}{\tilde{p}}_k\stackrel{(2)}{=}q{\tilde{p}}_k\epsilon .$$

$$x=a_1^{m_1}\cdots a_k^{m_k}=b_1^{n_1}\cdots b_l^{n_l}.$$

$$u=a_1^{m_1}\cdots a_{i-1}^{m_{i-1}}=b_1^{n_1}\cdots b_{i-1}^{n_{i-1}}.$$

Full text

1 Introduction

Computing proximities is a core task for many important time-series mining tasks such as similarity search, nearest-neighbor classification, and clustering. The choice of proximity measure affects the performance of the corresponding proximity-based method. Consequently, a plethora of task-specific proximity measures have been devised [1, 3].

One research direction in time series mining is devoted to constructing data-specific distance functions that can capture some of the data’s intrinsic structure. For example, the dynamic time warping (dtw) distance has been devised to account for local variations in speed [39, 31]. Further examples include distances and techniques that account for other variations such as variation in speed, amplitude, offset, endpoints, occlusion, complexity, and mixtures thereof [5].

In some – partly influential – work, the different distances are ascribed as invariant under the respective variation. For example, the dtw-distance is considered as warping-invariant (invariant under variation in speed) [5, 10, 11, 18, 27, 35], the CID-distance [5] as complexity-invariant, and the $\psi$-dtw distance [35] as endpoint-invariant. Such statements in the literature collide with the mathematical meaning of invariance. In mathematics, a distance function $d$ is called invariant under some transformation $f$ if $d(x,y)=d(f(x),f(y))$ for all elements $x,y$ from some domain. According to the mathematical definition of invariance, the CID-distance is not complexity-invariant, the $\psi$-dtw distance is not endpoint-invariant, and in particular, the dtw-distance is not warping-invariant (see Fig. 1 and Section 2.3). To avoid confusion due to ambiguity, we assume the mathematical definition of invariance and say instead that the dtw-distance can cope with (or accounts for) temporal variation.

The literature postulates that the dtw-distance can cope with temporal variations but stores and processes time series in a form as if the dtw-distance cannot cope with such variations. For example, the left columns of Figure 1 shows two time series $x$ and $y$ from the UCR dataset SmallKitchenAppliances [12]. Both time series have length $720$ and mostly consist of constant segments of variable length (time). If the dtw-distance can cope with temporal variations, why do we not collapse constants segments to single points? The right column of Figure 1 shows the corresponding condensed forms $x^{*}$ and $y^{*}$ obtained by collapsing constant segments to singletons. The condensed forms have length $\mathop{\left\lvert x^{*}\right\rvert}=25$ and $\mathop{\left\lvert y^{*}\right\rvert}=35$, which is substantially shorter than the lengths of their original time series. As a consequence, the space-saving ratios are $\rho_{s}(x)=96.5\%$ and $\rho_{s}(y)=95.1\%$ and the speed-up factor of computing the dtw-distance $\delta(x^{*},y^{*})$ over $\delta(x,y)$ is $592.5$.

Even though Fig. 1 refers to one of the most extreme examples one can find in the UCR archive, sacrificing potential space-saving ratios and speed-up factors generally ask for a justification. An example of such a justification would be a better solution quality in time series mining tasks. Evidence to justify the inconsistency between the idea behind the dtw-distance and how this distance has been actually applied is missing.

We frame the inconsistency-problem into the wider perspective of the warping-invariance problem. Though the dtw-distance has been proposed to account for temporal variation potential effects caused by the lack of warping-invariance have not been clearly disclosed—leaving a number of open questions:

What are the negative effects contributed by the lack of warping-invariance? 2. 2.

How can we convert the dtw-distance to a warping-invariant distance? 3. 3.

What are the practical implications of a warping-invariant version of the dtw-distance?

Possible adverse effects contributed by the absence of warping-invariance would ask for a solution. In this case, a possible solution would aim at converting the dtw-distance to a warping-invariant distance. This in turn raises the question, whether such a solution is not only theoretically fancier but also has practical benefits.

To close the gap of the warping-invariance problem, we convert the dtw-distance to a warping-invariant semi-metric, called time-warp-invariant (twi) distance. The resulting twi-distance corresponds to the dtw-distance on the subset of condensed forms. The construction of the twi-distance answers the second question and forms the foundation to answer the first and third question.

To answer the first question, we show that the lack of warping-invariance contributes to an adverse effect of the nearest-neighbor rule in dtw-spaces. The nearest-neighbor rule is an important subroutine for several time series mining methods. Examples include similarity search, $k$-nearest-neighbor classification [1, 4], k-means clustering [30, 17], learning vector quantization [20, 38], and self-organizing maps [23]. The peculiarity of the nearest-neighbor rule affects all the time series mining methods just mentioned. As an example, we show that it is possible to arbitrarily increase the cluster quality of k-means without changing the clusters under certain conditions.

Finally, we answer the third question on the practical benefit of the twi-distance with respect to (i) computation time, (ii) storage consumption, and (iii) solution quality. The results show that the error rates of the twi and dtw nearest-neighbor are practically equivalent in a Bayesian sense, but the twi-distance consumes less storage and is faster to compute than the dtw-distance for a broad range of problems.

The rest of this article is structured as follows: Section 2 provides an informal overview of the main theoretical results. In Section 3, we point to peculiarities of the dtw-distance. Experiments are presented and discussed in Section 4. Finally, Section 5 concludes with a summary of the main results, their impact, and an outlook to further research. The theoretical treatment including all proof has been delegated to the appendix.

2 A Warping-Invariant DTW Distance

This section introduces the time-warp-invariant (twi) distance and shows that this distance is a warping-invariant semi-metric induced by the dtw-distance. We first introduce metric properties and the dtw-distance. Then we define the concept of warping-invariance. Then we present the main results to convert a dtw-space into a warping-invariant semi-metric space. Finally, we discuss some computational issues of the twi-distance. The theoretical treatment is delegated to A.

2.1 Metric Properties

In this section, we introduce metric properties and define the notion of semi-metric and pseudo-metric. A metric on a set ${\mathcal{X}}$ is a function $d:{\mathcal{X}}\times{\mathcal{X}}\rightarrow{\mathbb{R}}$ satisfying the following properties:

$d(x,y)\geq 0$ (non-negativitiy) 2. 2.

$d(x,y)=0\;\Leftrightarrow\;x=y$ (identity of indiscernibles) 3. 3.

$d(x,y)=d(y,x)$ (symmetry) 4. 4.

$d(x,z)\leq d(x,y)+d(y,x)$ (triangle inequality)

for all $x,y,z\in{\mathcal{X}}$. We call $d$ a distance function. A semi-metric is a distance function satisfying the first three metric properties $(1)-(3)$. A pseudo-metric is a distance function satisfying the metric properties $(1),(3),$ and $(4)$.

2.2 The DTW-Distance

For a given $n\in{\mathbb{N}}$, we write $[n]=\mathop{\left\{1,\ldots,n\right\}}$. A time series of length $n$ is a sequence $x=(x_{1},\ldots,x_{n})$ with elements $x_{i}\in{\mathbb{R}}$ for all $i\in[n]$. We denote the set of all time series of finite length by ${\mathcal{T}}$. Consider the ($m\times n$)-grid defined by

[TABLE]

A warping path of order $m\times n$ and length $\ell$ is a sequence $p=(p_{1},\dots,p_{\ell})$ through the grid $[m]\times[n]$ consisting of $\ell$ points $p_{l}=(i_{l},j_{l})\in[m]\times[n]$ such that

$p_{1}=(1,1)$ and $p_{\ell}=(m,n)$ (boundary conditions) 2. 2.

$p_{l+1}-p_{l}\in\mathop{\left\{(1,0),(0,1),(1,1)\right\}}$ for all $l\in[\ell-1]$ (step condition)

We denote the set of all warping paths of order $m\times n$ by ${\mathcal{P}}_{m,n}$. Figure 2(a) depicts examples of warping paths through a ($3\times 3$)-grid.

Suppose that $p=(p_{1},\ldots,p_{\ell})\in{\mathcal{P}}_{m,n}$ is a warping path with points $p_{l}=(i_{l},j_{l})$ for all $l\in[\ell]$. Then $p$ defines an (warping) of time series $x=(x_{1},\ldots,x_{m})$ and $y=(y_{1},\ldots,y_{n})$ to time series $\phi_{p}(x)=(x_{i_{1}},\ldots,x_{i_{\ell}})$ and $\psi_{p}(y)=(y_{j_{1}},\ldots,y_{j_{\ell}})$ of the same length $\ell$. By definition, the length $\ell$ of a warping path satisfies $\max(m,n)\leq\ell<m+n$. This shows that $\ell\geq\max(m,n)$ and therefore $\phi_{p}(x)$ and $\psi_{p}(y)$ are indeed expansions of $x$ and $y$. Figure 2(b) illustrates the concept of expansion.

The cost of warping time series $x$ and $y$ along warping path $p$ is defined by

[TABLE]

where $\mathop{\left\lVert\cdot\right\rVert}$ denotes the Euclidean norm and $\phi_{p}$ and $\psi_{p}$ are the expansions defined by $p$. Then the dtw-distance of $x$ and $y$ is of the form

[TABLE]

A warping path $p$ with $C_{p}(x,y)=\operatorname{\delta}^{2}(x,y)$ is called an optimal warping path of $x$ and $y$. By definition, the dtw-distance minimizes the Euclidean distance between all possible expansions that can be derived from warping paths. Computing the dtw-distance and deriving an optimal warping path is usually solved by applying a dynamic program [31, 39].

The dtw-distance satisfies only the metric properties $(1)$ and $(3)$. Thus, the dtw-distance is neither a metric, nor a semi- and pseudo-metric.

2.3 Warping-Invariance

In this section, we define the concept of warping-invariance. For this, we introduce the notions of expansion and compression. Expansions have been introduced in the previous section in the context of warping. Here, we introduce expansions and compressions as operations on time series.

An expansion of a time series is obtained by replicating a subset of its elements. More precisely, suppose that $x=(x_{1},\ldots,x_{n})$ is a time series. A time series $x^{\prime}$ is an expansion of time series $x$, written as $x^{\prime}\succ x$, if there are indexes $i_{1},\ldots,i_{k}\in[n]$ and positive integers $r_{1},\ldots,r_{k}\in{\mathbb{N}}$ for some $k\in{\mathbb{N}}_{0}$ such that $x^{\prime}$ is of the form

[TABLE]

By setting $k=0$ we find that a time series is always an expansion of itself. A time series $x$ is a compression of time series $x^{\prime}$, written as $x\prec x^{\prime}$, if $x^{\prime}$ is an expansion of $x$. A time series $z$ is a common compression of time series $x$ and $y$ if $z$ is a compression of $x$ and of $y$, that is $z\prec x$ and $z\prec y$. We write $x\sim y$ if both time series have a common compression. Figure 3 depicts the relationships between time series that have a common compression in form of a directed rooted tree of infinite depth. The root is the common compression of all nodes in the tree.

A distance function $d:{\mathcal{T}}\times{\mathcal{T}}\rightarrow{\mathbb{R}}_{\geq 0}$ is warping-invariant if

[TABLE]

for all time series $x,y,x^{\prime},y^{\prime}\in{\mathcal{T}}$ with $x\sim x^{\prime}$ and $y\sim y^{\prime}$. Observe that warping-invariance as defined here is more general than the notion of invariance introduced in Section 1. The difference is that invariance in Section 1 assumes that both time series undergo the same transformation, whereas in this section the transformation can be different for both time series. The next example shows that the dtw-distance is not warping-invariant.

Example 1**.**

Let $x=(0,1)$ and $x^{\prime}=(0,1,1)$ be two time series. Then $x^{\prime}$ is an expansion of $x$ and $\operatorname{\delta}(x,x^{\prime})=0$. Suppose that $y=(0,2)$ is another time series. Then we have

[TABLE]

This implies $\operatorname{\delta}(x,y)\neq\operatorname{\delta}(x^{\prime},y)$, where $x$ is a common compression of $x,x^{\prime}$ and $y$ is a compression of itself. Hence, the dtw-distance is not warping-invariant. $\square$

Warping-invariance as introduced here must not be confused with the notion of warping-invariance and related terms in the literature [5, 10, 11, 27, 35]. In [18], I falsely insinuated that the dtw-distance is warping-invariant. There are also statements closely related to the notion of warping-invariance that are prone to misunderstandings. For example:

• •

The sequences are “warped” non-linearly in the time dimension to determine a measure of their similarity independent of certain non-linear variations in the time dimension [22].

• •

Dynamic time warping (DTW) was introduced […] to overcome this limitation [of the Euclidean distance] and give intuitive distance measurements between time series by ignoring both global and local shifts in the time dimension [32].

As Example 1 indicates, the dtw-distance is neither independent of non-linear variations nor ignores global and local shifts in the time dimension.

2.4 The Time-Warp-Invariant Distance

In this section, we present the main results of the theoretical exposition. In particular, we introduce the time-warp-invariant (twi) distance and show that the twi-distance is a warping-invariant semi-metric. A contains the full theoretical treatment including all proofs.

The blueprint to convert a dtw-space into a semi-metric space follows the standard approach to convert a pseudo-metric space into a metric space. However, additional issues need to be resolved, because the dtw-distance satisfies less metric properties than a pseudo-metric. According to the blueprint, we proceed as follows: First, we introduce an equivalence relation $\sim$ on time series. Then we construct the quotient space of $\sim$. Next, we endow the quotient space with a quotient distance, called twi-distance. Finally, we show that the twi-distance is a warping-invariant semi-metric.

We begin with introducing an equivalence relation on time series. Two time series are said to be warping-identifiable if their dtw-distance is zero. Formally, warping identification is the relation defined by $x\sim y\,\Leftrightarrow\,\operatorname{\delta}(x,y)=0$ for all time series $x,y\in{\mathcal{T}}$. From A follows that warping identification and common compression are equivalent definitions of the same relation. In light of Fig. 3, this suggests that all pairs of time series from the same tree are warping-identifiable. The next result shows that warping identification is indeed an equivalent relation and thereby forms the foundation for converting the dtw-distance to a semi-metric.

Proposition 2**.**

Warping identification $\sim$ is an equivalence relation on ${\mathcal{T}}$.

The assertion of Prop. 2 is not self-evident, because warping paths are not closed under compositions and the dtw-distance fails to satisfy the triangle inequality. Therefore, it might be the case that there are time series $x,y,z$ such that $\delta(x,y)=\delta(y,z)=0$ but $\delta(x,z)>\delta(x,y)+\delta(y,z)=0$.

We continue with constructing a new set, the so-called quotient space of equivalence relation $\sim$. This task consists of two steps. The first step forms equivalence classes and the second step collects all equivalence classes to a new set. This new set is the desired quotient space.

The equivalence class of a time series $x$ is the set

[TABLE]

consisting of all time series that are warping-identifiable to $x$. Time series from the same equivalence class have zero dtw-distance and time series from different equivalence classes have non-zero dtw-distance. Figure 4 depicts examples of two different equivalence classes.

The set

[TABLE]

of all equivalence classes is the quotient space of ${\mathcal{T}}$ by warping identification $\sim$. With regard to the visual representation of Fig. 4, the quotient space consists of all trees rooted at non-compressible time series.

Next, we endow the quotient space ${\mathcal{T}}^{*}$ with a distance. The twi-distance is defined by

[TABLE]

Readers unfamiliar with the concept of infimum may safely replace it by the minimum in this particular case. The twi-distance is a dtw-distance between special sets, the equivalence classes under warping identification. As a distance on sets, the twi-distance between two equivalence classes is the infimum dtw-distance between their corresponding representatives. For example, the twi-distance between the equivalence classes represented by the trees in Fig. 4 is the infimum of all dtw-distances between time series obtained by expanding $x^{*}$ and time series obtained by expanding $y^{*}$.

The last step consists in showing that the twi-distance is a warping-invariant semi-metric. However, the twi-distance as defined in Eq. (1) introduces an additional problem. The naive way to compute the twi-distance by comparing infinitely many dtw-distances between time series from either equivalence class is computationally unfeasible.

To show warping-invariance, semi-metric properties, and to overcome the problem of an incomputable form, we derive an equivalent formulation of the twi-distance. For this, we introduce irreducible time series. A time series is said to be irreducible if it cannot be expressed as an expansion of a shorter time series. For example, the roots of the trees in Fig. 3 and 4 are all irreducible time series. The next result shows that an equivalence class can be generated by expanding an irreducible time series in all possible ways.

Proposition 3**.**

For every time series $x$ there is an irreducible time series $x^{*}$ such that

[TABLE]

The irreducible time series $x^{*}$ in Prop. 3 is called the condensed form of $x$. Every time series has a unique condensed form by A, Prop. 11. Hence, $x^{*}$ is the condensed form of all time series contained in the equivalence class $[x]$. Proposition 3 states that the equivalence classes are the trees rooted at a their respective condensed forms as illustrated in Figure 3 and 4. Next, we show that expansions do not decrease the dtw-distance to other time series.

Proposition 4**.**

Let $x,x^{\prime}\in{\mathcal{T}}$ such that $x^{\prime}\succ x$. Then $\operatorname{\delta}(x^{\prime},z)\geq\operatorname{\delta}(x,z)$ for all $z\in{\mathcal{T}}$.

A special case of Prop. 4 has been proved in [8]. We will use Prop. 4 in Section 3 to discuss peculiarities of data-mining applications on time series. By invoking Prop. 3 and Prop. 4 we obtain the first key result of this contribution.

Theorem 5**.**

The twi-distance $\operatorname{\delta}^{*}$ induced by the dtw-distance $\operatorname{\delta}$ is a well-defined semi-metric satisfying

[TABLE]

for all $x,y\in{\mathcal{T}}$ with condensed forms $x^{*}$ and $y^{*}$, respectively.

Theorem 5 converts the dtw-distance to a semi-metric and shows that the twi-distance can be efficiently computed as discussed in Section 2.5.

The second key result of this contribution shows that the semi-metric induced by the dtw-distance is warping-invariant. Since warping-invariance is defined on the set ${\mathcal{T}}$ rather than on the quotient space ${\mathcal{T}}^{*}$, we extend the twi-distance $\operatorname{\delta}^{*}$ to a distance on ${\mathcal{T}}$ by virtue of

[TABLE]

We call $\operatorname{\delta}^{\sim}$ the canonical extension of $\operatorname{\delta}^{*}$.

Theorem 6**.**

The canonical extension $\operatorname{\delta}^{\sim}$ of the twi-distance $\operatorname{\delta}^{*}$ is warping-invariant.

2.5 Computational Issues

In this section, we discuss computational issues of the twi-distance and point to optimization techniques in nearest-neighbor search.

Computing the TWI-Distance.

The twi-distance and dtw-distance only coincide on irreducible time series. If at least one of the two time series to be compared is not irreducible, there are two ways to compute the twi-distance:

The traditional way assumes the existence of time series that are not irreducible. Then the twi-distance $\operatorname{\delta}^{*}([x],[y])$ can be computed by first compressing $x$ and $y$ to their condensed forms $x^{*}$ and $y^{*}$, resp., in linear time and then taking their dtw-distance $\operatorname{\delta}(x^{*},y^{*})$. 2. 2.

From the perspective of applications that demand warping-invariance, there is no point of storing all but one element of a constant sub-sequence. Instead, one assumes that all time series are irreducible and then computes the twi-distance as an ordinary dtw-distance on irreducible time series.

Figure 5 illustrates how to compute the twi-distance for the case that both time series to be compared are not irreducible. Let $\mathop{\left\lvert x\right\rvert}$ denote the length of a time series $x$. Suppose that $x$ and $y$ are two time with corresponding condensed forms $x^{*}$ and $y^{*}$, respectively. Then $\rho_{\text{c}}(x)=\mathop{\left\lvert x\right\rvert}/\mathop{\left\lvert x^{*}\right\rvert}$ is the compression-ratio of $x$. The higher the compression-ratio of $x$, the shorter is its condensed form $x^{*}$. The speed-up factor of the twi-distance over the dtw-distance is given by

[TABLE]

where $\phi_{1}$ assumes that time series are not necessarily irreducible and $\phi_{2}$ assumes that all time series are irreducible. Suppose that $n=\mathop{\left\lvert x\right\rvert}=\mathop{\left\lvert y\right\rvert}$ and $m=\max\mathop{\left\{\mathop{\left\lvert x^{*}\right\rvert},\mathop{\left\lvert y^{*}\right\rvert}\right\}}$. Then the first speed-up factor $\phi_{1}$ satisfies

[TABLE]

if $n>m$. Thus, computing the twi-distance is more efficient than computing the dtw-distance once there is at least one replication in both time series $x$ and $y$. In particular, the twi-distance is much more efficient to compute than the dtw-distance for extremely long time series with high compression-ratio. Such time series occur, for example, when measuring electrical power of household appliances with a sampling rate of a few seconds collected over several months, twitter activity data sampled in milliseconds, and human activities inferred from a smart home environment [27]. For such time series, storage consumption can be reduced to over $90\%$.

Recently, two algorithms have been devised to cope with time series with high compression-ratio. The first algorithm is AWarp proposed by [27]. This algorithm exactly computes the dtw-distance for time series consisting of binary values and is an approximation of the dtw-distance otherwise. The second algorithm is the bdtw-distance (block-dtw) proposed by [34]. The bdtw-algorithm generalizes AWarp by exactly computing the dtw-distance not only for binary but also for any two-valued time series. For time series with more than two-values, the bdtw-distance is an approximation of the dtw-distance and comes with a lower and upper bound.

Both algorithms have a similar speed-up factor as the twi-distance but are slower by a small factor. This factor is caused by additional operations such as multiplying every element with the number of its repetitions in order to approximate the dtw-distance.

Optimizing Nearest-Neighbor Search.

One common task in time series mining is nearest-neighbor search [36]. Given a dataset ${\mathcal{D}}$ consisting of $N$ time series and a query time series $q$, the goal is to find a time series $x\in{\mathcal{D}}$ that is closest to $q$ with respect to the dtw-distance. A naive implementation compares the query to all $N$ time series to identify the nearest-neighbor. The naive approach is unfeasible for large $N$ and for long time series due to the quadratic complexity of the dtw-distance.

As a consequence, nearest-neighbor search has been continually optimized by devising combinations of sophisticated pruning methods such as lower bounding and early abandoning [36]. Since the twi-distance is a dtw-distance on irreducible time series, all pruning methods applicable to the dtw-distance can also be applied to the twi-distance. In some cases, we need to ensure that the irreducible time series are of the same length. This holds for lower-bounding techniques such as LB_Lemire [24]and LB_Keogh [21]. We can align irreducible time series to identical length using any technique that preserves irreducibility. An example of an irreducibility-preserving technique is linear interpolation.

3 Peculiarities of the DTW-Distance

In this section, we discuss some peculiarities of the dtw-distance, which do not occur for the twi-distance.

3.1 The Effect of Expansions on the Nearest-Neighbor Rule

Suppose that $x^{\prime}$ is an expansion of $x$. Then Prop. 4 yields

[TABLE]

for all time series $y$. Equation (2) states that a time series is warping identical with its expansions. Equation (3) says that expansions do not decrease the dtw-distance to other time series. Equation (3) affects the nearest-neighbor rule. To see this, we consider a set ${\mathcal{D}}=\mathop{\left\{x,y\right\}}\subseteq{\mathcal{T}}$ of two prototypes. The Voronoi cells of ${\mathcal{D}}$ are defined by

[TABLE]

Suppose that $z$ is a time series. If $z\in{\mathcal{V_{D}}}(x)$ then prototype $x$ is the nearest-neighbor of $z$. Otherwise, prototype $y$ is the nearest-neighbor of $z$. To break ties, we arbitrarily assign $x$ as nearest-neighbor for all time series from the boundary ${\mathcal{V_{D}}}(x)\cap{\mathcal{V_{D}}}(y)$. The nearest-neighbor rule assigns $z$ to its nearest-neighbor.

Suppose that we replace prototype $x$ by an expansion $x^{\prime}$ to obtain a modified set ${\mathcal{D}}^{\prime}=\mathop{\left\{x^{\prime},y\right\}}$ of prototypes. Although $x^{\prime}$ and $x$ are warping identical by Eq. (2), the Voronoi cells of ${\mathcal{D}}$ and ${\mathcal{D}}^{\prime}$ differ. From Eq. (3) follows that ${\mathcal{V_{D^{\prime}}}}(x^{\prime})\subseteq{\mathcal{V_{D}}}(x)$, which in turn implies ${\mathcal{V_{D^{\prime}}}}(y)\supseteq{\mathcal{V_{D}}}(y)$. Keeping prototype $y$ fixed, expansion of prototype $x$ decreases its Voronoi cell and thereby increases the Voronoi cell of $y$. This shows that the nearest-neighbor rule in dtw-spaces depends on temporal variation of the prototypes.

This peculiarity of the nearest-neighbor rule affects time series mining methods in dtw-spaces such as similarity search, $k$-nearest-neighbor classification, k-means clustering, learning vector quantization and self-organizing maps.

3.2 The Effect of Expansions on k-Means

As an example, we discuss how the peculiarities of the nearest-neighbor rule affect the k-means algorithm. Box 6 provides a brief review about k-means in dtw-spaces.

To apply k-means in dtw-spaces, we need a concept of average of time series. Different forms of time series averages have been proposed (see [33] for an overview). One principled formulation of an average is based on the notion of Fréchet function [15]: Suppose that ${\mathcal{S}}=\mathop{\left(x_{1},\dots,x_{n}\right)}$ is a sample of $n$ time series $x_{i}\in{\mathcal{T}}$. Then the Fréchet function of ${\mathcal{S}}$ is defined by

[TABLE]

A sample mean of ${\mathcal{S}}$ is any time series $\mu\in{\mathcal{T}}$ that satisfies $F(\mu)\leq F(z)$ for all $z\in{\mathcal{T}}$. A sample mean exists but is not unique in general [19]. Computing a mean of a sample of time series is NP-hard [9]. Efficient heuristics to approximate a mean of a fixed and pre-specified length are stochastic subgradient methods [33] and majorize-minimize algorithms [17, 28].

Next, we generalize the k-means algorithm to dtw-spaces by replacing arithmetic means of vectors with sample means of time series. For this, we first introduce some notations. A partition of a set ${\mathcal{S}}=\mathop{\left\{x_{1},\dots,x_{n}\right\}}\subseteq{\mathcal{T}}$ of time series is a set ${\mathcal{C}}=\mathop{\left({\mathcal{C}}_{1},\ldots,{\mathcal{C}}_{k}\right)}$ of $k$ non-empty subsets ${\mathcal{C}}_{i}\subseteq{\mathcal{S}}$, called clusters, such that ${\mathcal{S}}$ is the disjoint union of these clusters. By $\Pi_{k}$ we denote the set of all partitions consisting of $k$ clusters. The objective of k-means is to minimize the cost function

[TABLE]

where $x\in{\mathcal{C}}_{i}$ if the centroid $\mu_{i}$ of cluster ${\mathcal{C}}_{i}$ is the nearest-neighbor of $x$ for all $i\in[k]$. By definition of the cost function $J$, the centroid $\mu_{i}$ of cluster ${\mathcal{C}}_{i}$ coincides with the sample mean of ${\mathcal{C}}_{i}$. Thus, we can equivalently express the cost function $J$ by

[TABLE]

where $F_{i}$ is the Fréchet function of cluster ${\mathcal{C}}_{i}$. To minimize the cost function $J$, the standard k-means algorithm starts with an initial set $\mu_{1},\ldots,\mu_{k}$ of centroids and then proceeds by alternating between two steps until termination:

Assignment step: Assign each sample time series to the cluster of its closest centroid. 2. 2.

Update step: Recompute the centroids for every cluster.

Due to non-uniqueness of sample means, the performance of k-means does not only depend on the choice of initial centroids as in Euclidean spaces, but also on the choice of centroids in the update step. In the following, we present two examples that illustrate a peculiar behavior of k-means in dtw-spaces.

Example 7**.**

We assume that the four time series

[TABLE]

are partitioned into two clusters ${\mathcal{C}}_{1}=\mathop{\left\{x_{1},x_{2}\right\}}$ and ${\mathcal{C}}_{2}=\mathop{\left\{x_{3},x_{4}\right\}}$. Cluster ${\mathcal{C}}_{1}$ has a unique mean $\mu_{1}=(-1,0,1)$. Cluster ${\mathcal{C}}_{2}$ has infinitely many means as indicated by Figure 7. For every $r\in{\mathbb{N}}$ the time series

[TABLE]

is a mean of ${\mathcal{C}}_{2}$. From Eq. (2) and the transitivity of expansions follows that $\mu_{2}^{r}$ and $\mu_{2}^{s}$ are warping identical for all $r,s\in{\mathbb{N}}$. Equation (3) yields $\operatorname{\delta}(\mu_{2}^{1},y)\leq\cdots\leq\operatorname{\delta}(\mu_{2}^{r},y)$ for all time series $y\in{\mathcal{T}}$ and all $r\in{\mathbb{N}}$. Hence, with increasing number $r$ of replications, the Voronoi cell of site $\mu_{2}^{r}$ decreases, whereas the Voronoi cell of site $\mu_{1}$ increases. $\square$

Example 7 shows that the Voronoi cell of a centroid $\mu_{i}$ can be externally controlled by expanding or compressing $\mu_{i}$ without changing the Fréchet variation $F(\mu_{i})$. This, in turn, affects the assignment step, which is based on the nearest-neighbor rule.

Example 8**.**

Consider the two clusters of the four time series from Example 7. One way to measure the quality of the clustering ${\mathcal{C}}=\mathop{\left\{{\mathcal{C}}_{1},{\mathcal{C}}_{2}\right\}}$ is by cluster cohesion and cluster separation. Cluster cohesion is typically defined by

[TABLE]

where $F_{1}(z)$ and $F_{2}(z)$ are the Fréchet functions of ${\mathcal{C}}_{1}$ and ${\mathcal{C}}_{2}$, respectively. Thus, cluster cohesion sums the Fréchet variations within each cluster. Cluster separation can be defined by

[TABLE]

Cluster separation measures how well-separated or distinct two clusters are. A good clustering simultaneously minimizes cluster cohesion and maximizes cluster separation. Cluster cohesion is invariant under the choice of cluster means, because the Fréchet variations $F_{1}(\mu_{1})$ and $F_{2}(\mu_{2})$ are well-defined. The situation is different for cluster separation. From Eq. (2) follows $\operatorname{\delta}(\mu_{1},\mu_{2}^{1})^{2}\leq\cdots\leq\operatorname{\delta}(\mu_{1},\mu_{2}^{r})$ for all $r\in{\mathbb{N}}$. Thus, cluster separation depends on the choice of mean $\mu_{2}$ of the second cluster ${\mathcal{C}}_{2}$ (the mean $\mu_{1}$ of ${\mathcal{C}}_{1}$ is unique). Figure 8 shows that cluster separation linearly increases with increasing number $r$ of replications.

$\square$

Example 8 illustrates that cluster cohesion is independent of the choice of mean but cluster separation can be maximized to infinity by expansions. In the given setting, cluster quality using cohesion and separation is not an inherent property of a clustering but rather a property of the chosen means as centroids, whose lengths can be controlled externally. Finally, we note that the scenario described in Example 8 is not common but rather is exceptional [8]. Nevertheless, subject to certain conditions, we can arbitrarily increase cluster quality without changing the clusters.

4 Experiments

In this experiment, we test the hypothesis of whether the twi-distance is more efficient than the dtw-distance without sacrificing for solution quality for a broad range of problems in nearest-neighbor classification. For this, we assess (i) the prevalence of reducible (non-irreducible) time series, (ii) the computation times, and (ii) the error-rates in nearest-neighbor classification

4.1 UCR Benchmark Data

We used the datasets from the 2015 – 2018 version of the UCR Time Series Classification Repository [12].

4.2 Storage Consumption

In this section, we first study the prevalence of reducible time series. The prevalence of reducible time series provides us information on whether the twi-distance is suitable only for exceptional cases or of broader interest. To answer this question, we computed the condensed form of every time series. A time series is reducible if it is longer than its condensed form. For every dataset, we recorded the percentage of reducible time series and the average number of deleted replications over the subset of reducible time series.

Figure 9(a) shows the estimated cumulative distribution function of the percentage $p_{\text{red}}$ of reducible time series. The total number of time series over all $85$ datasets is $N=135,\!793$. The percentage of reducible time series over all $N$ time series is $67.8\%$. All but two out of $85$ datasets contain reducible time series. There are five datasets with at most $1\%$ reducible time series, and $17$ datasets with at least $99\%$ reducible time series. Among the reducible time series, the corresponding condensed forms are approximately $28.0\%$ shorter on average. These results indicate that reducible time series are not an exceptional phenomenon but rather occur frequently. Consequently, application of the twi-distance is not restricted to exceptional cases but to a broad range of problems.

Next, we study the relationship between the length $\ell$ of time series and the percentage $p_{\text{red}}$ of reducible time series. Figure 9(b) shows a scatter plot of both quantities. We are interested to quantify the linear and the monotonic relationship between both variables. The Pearson correlation coefficient is $\rho=0.32$ indicating a weak positive linear correlation. Figure 9(b) however suggests that this correlation could be caused by outliers. The Spearman and Kendall correlation coefficients are $r=0.28$ and $\tau=0.18$, resp., indicating a weak to negligible monotonic positive monotonic correlation. We intuitively might expect that it is more likely to see two consecutive identical values in longer rather than in shorter time series. The results however do not convincingly support this intuition.

4.3 Computation Time

In this experiment, we compare the computation time of the twi-distance against other distances.

Experimental Protocol.

We used the following distance functions: Euclidean (euc), dtw, bdtw, and twi.111We discuss the block dtw (bdtw) distance in Section 2.5. We excluded the StarLightCurves and UWaveGestureLibraryAll datasets for computational reasons. From every of the remaining $83$ UCR datasets, we randomly selected $100$ pairs of time series. Then we computed the distance of each pair $100$-times and recorded the average time giving a total of $83\times 100$ average times for every distance function.

Results and Discussion.

Figure 10 plots the computation times of every distance measure as a function of the (average) space-saving ratio. The computation times are shown as speed-up factors of all distances over the dtw-distance (left) and of the twi-distance over the bdtw-distance (right). The space-saving ratio for two time series $x$ and $y$ with condensed forms $x^{*}$ and $y^{*}$, resp., is defined by

[TABLE]

The shorter the condensed forms the more space can be saved with respect to the original time series.

First, note that computing the twi-distance is never slower than computing the dtw- or bdtw-distance by construction. Variations in time, however, occur due to Java’s just-in-time compilation. The speed-up factors of twi and bdtw over the dtw-distance grow with increasing space-saving ratio. At very high space-saving ratios computing the twi-distance is up to $6,000$ times faster than computing the dtw-distance and is even faster than computing the Euclidean distance. In addition, computing the twi-distance is about $2$ to $4$ times faster than computing the bdtw-distance. Computing the bdtw-distance is slower, because additional operations are required to approximate the dtw-distance. The results suggest that the twi-distance is a more efficient alternative to the dtw-distance.

4.4 Nearest-Neighbor Classification

In this experiment, we assess the performance of the twi-distance in nearest-neighbor (nn) classification.

Experimental Protocol.

We excluded the StarLightCurves and UWaveGestureLibraryAll datasets for computational reasons. We applied $10$-fold cross-validation on the remaining $83$ datasets and recorded the average accuracy over all folds. Bayesian hypothesis testing was used to ensure statistical validity of the results. We compared the following distance functions to nn classification: Euclidean (euc), dtw, bdtw${}_{\text{lb}}$, bdtw${}_{\text{ub}}$, and twi, where bdtw${}_{\text{lb}}$ and bdtw${}_{\text{ub}}$ refer to the lower and upper bound of bdtw, respectively. Note that this experimental protocol does not follow the standard protocol in the literature. In Box 11, we point to arguments why we applied a different protocol.

Results and Discussion.

Figure 13 shows the pairwise posteriors for the Bayesian sign-rank test with a rope of $0.5\%$. Box 12 briefly reviews Bayesian hypothesis testing and explains how to interpret the results in Figure 13. Table 1 summarizes the percentages of pairwise practical wins and practical losses over all $83$ datasets. The meanings of practical wins and losses follow from the explanations in Box 12.

There are two main observations from the results of the Bayesian test. First, warping-based nn-classifiers are practically better than the euc-nn classifier with more than $98\%$ probability. Second, the warping-based nn-classifiers are practically equivalent. Table 1 shows that the euc-nn classifier is still a good and fast alternative for many problems and that the twi-nn classifier has slight but not statistically validated advantages over all warping-based nn-classifiers.

The results are interesting because they challenge common practice in applying the dtw-distance: On the one hand, we postulate that the dtw-distance can cope with temporal variations. On the other hand, we store and process reducible time series with constant segments as if the dtw-distance is not capable to cope with temporal variation. This inconsistency in applying the dtw-distance sacrifices space-saving and computational benefits without gaining substantially improved error rates in general. The dtw-nn classifier is better than the twi-nn classifier by a margin of at least $1\%$ only on four datasets with a maximum of $2.5\%$ on Lighting2. Thus, there are a few datasets for which the dtw-distance should be preferred over the twi-distance. Consequently, the practical inconsistency of the dtw-distance lacks a general rationale in the particular case of nearest-neighbor classification.

4.5 Power Consumption Classification

In this experiment, we assess the performance of the twi-nn classifier on power consumption data consisting of long time series with large constant segments.

Data.

The 2nd version of the Almanac of Minutely Power dataset (AMPds2) provided by [25] includes electricity measurements of a house in the Greater Vancouver metropolitan area in British Columbia (Canada).222URL of AMPds2: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/FIE0S4. The dataset contains electricity measurements over a period of two years at one minute intervals from $21$ power sub-meters giving a total of $1,051,200$ readings from each sub-meter. In this study, we consider the real power P measured in VAR.

Distance Functions.

We applied the following distance functions to nearest-neighbor classification: euc, dtw, bdtw${}_{\text{lb}}$, bdtw${}_{\text{ub}}$, twi, opt-dtw, and opt-twi. The opt-distances optimize nearest-neighbor search by applying the following techniques: (i) Sakoe-Chiba band [31] of radius $0.1$, (iii) lower-bounding with LB_Keogh [21] followed by LB_Lemire [24], and (iii) early abandoning during dtw-computation. To apply the lower-bounding techniques, query and candidate time series should be of the same length. We aligned the candidates to the query either by truncation or expansion.

Experimental Protocol.

The task is to assign segments of power readings to their corresponding sub-meters. The sub-meters form $21$ classes. We split the 100K readings of each sub-meter $s$ into sequences lasting $d$ days, where $d\in\mathop{\left\{1,7,14,30\right\}}$ giving a total of $N_{d}(s)$ time series $x_{1}(s),\ldots,x_{N_{d}}(s)$. We partitioned the $N_{d}$ time series into a training and test set. Time series with odd index went into the training set and all other time series went into the test set. Table 2 shows the length of the time series and the sizes of the training and tests. For every $d$ and every distance measure, we applied the nearest-neighbor classifier and recorded the error rate and the total time to classify all test examples.

Results and Discussion.

Table 3 shows the error rates and computation times of the nearest-neighbor classifiers. Note that the computation times are wall-clock times taken on a multi-user machine and are therefore only rough upper bounds of the actual computation times.

The first observation is that the Euclidean distance and the dtw-distance are unsuitable for the given classification problem. The Euclidean distance is fast to compute (a minute up to half an hour) but results in too high error rates ($50\%-75\%$), whereas the dtw-distance results in substantially lower error rates ($6\%-20\%$) but is slow to compute (almost three days). Even the opt-dtw-nn classifier requires up to one and a half day to complete its computations which is about three times slower than the next slowest classifier (bdtw-nn).

The second observation is that the error rates of the warping-based classifiers are comparable with slight but not statistically validated advantages of more than $0.5\%$ for the bdtw${}_{\text{lb}}$-nn classifier on the $14$ days dataset.

The third observation is that the twi-nn and opt-twi-nn classifiers are fastest among the warping-based classifiers. The twi-nn classifier is about twice faster than the bdtw-nn classifiers, about four to five times faster than the opt-dtw-nn classifier and more than ten times faster than the dtw-classifier. The speed-up factor of the opt-twi-nn classifier over the twi-nn classifier is $3$. The space saving factor of twi over dtw is $73.8\%$ (see Table 2).

These findings essentially confirm the findings on the UCR datasets and suggest that the twi-distance is a fast and space-saving alternative to the dtw-distance and its bdtw-approximations without substantially sacrificing solution quality. In addition, the results indicate that even optimized nearest-neighbor search remains an open issue for warping-based distances. In such cases, we could resort to heuristics such as FastDTW [32] that can be also applied to irreducible time series, that is as approximations to the twi-distance.

4.6 Limitations of the DTW- and TWI-Distance

In this section, we study basic limitations of the dtw- and twi-distance in nearest-neighbor classification.

Data.

We generated a synthetic dataset ${\mathcal{D}}$ consisting of $200$ time series of length $100$ from two classes ${\mathcal{Y}}=\mathop{\left\{\pm 1\right\}}$ such that each class has $100$ elements. Figure 14 shows examples of time series from both classes and their corresponding condensed forms. The elements $x_{i}$ of a time series $x=(x_{1},\ldots,x_{100})\in{\mathcal{D}}$ of class $y\in{\mathcal{Y}}$ are of the form

[TABLE]

where $b_{y}$ is the class-dependent base value of the constant segment, $c_{y}$ is the class-dependent height of the cylinder, ${\mathcal{I}}(x)\subset[100]$ is a randomly selected interval of fixed length $10$, and $\varepsilon_{b},\varepsilon_{c}\in U(\theta)$ are uniformly distributed noise terms with values from the interval $[0,\theta]$.

Experimental Protocol.

A single experiment generates a dataset according to Eq. (6) and then splits the dataset into a train and test set with balanced class distribution. Finally, we applied the nearest-neighbor classifier using the euc-, dtw-, and twi-distance. We repeated this experiment $50$-times and reported the average error rates of each classifier.

Results and Discussion.

Table 4 presents the results for five different configurations of the data generating parameters. The overall observation is that the euc-nn classifier failed because it is sensitive to temporal variations of the cylinder.

Experiment E1 & E2: The twi-nn classifier failed in experiment E1 with an error rate of $37\%$, because the base value $b_{y}$ of the constant segment is the discriminative feature to distinguish between both classes but the noise in the cylinder contributed most to the twi-distance. In contrast, the dtw-distance perfectly separated both classes because the accumulated differences between the constant segments from different classes contributed most to the distance and outweighed the noisy differences of the cylinders. These findings indicate that the twi-distance can fail when differences in constant segments matter. In light of the failure of the euc-nn classifier, this experiment provides an example for which warping is important but warping-invariance is not desired.

The noise $\varepsilon_{c}$ in experiment E1 is high because it can take values up to $100\%$ of the cylinder height $c_{y}$. In experiment E2, we used the same configuration as in experiment E1 but reduced the maximum contribution of the noise term $\varepsilon_{c}$ to $10\%$ of $c_{y}$. The results show that both warped-nn classifiers perfectly separate both classes. The findings of both experiments indicate that the twi-nn classifier is more sensitive to noise than the dtw-nn classifier given that the base value $b_{y}$ substantially contribute as discriminant between both classes.

Experiment E3 & E4: In experiment E1 & E2, the base value $b_{y}$ was the discriminative feature, whereas the cylinder heights $c_{y}$ were identical. In experiments E3 & E3, we reversed the roles of $b_{y}$ and $c_{y}$. In experiment E3, the dtw-nn classifier failed with an error rate of $31\%$ and the twi-nn classifier perfectly separated both classes due to the opposite reasons as in experiment E1. These results suggest that the dtw-distance can fail when differences in constant segments are irrelevant. This experiment provides an example for which warping alone is insufficient but warping-invariance matters.

In experiment E4, we reduced the noise $\varepsilon_{b}$ to $\theta_{b}=0.1$. For similar reasons as in experiment E2, we observe that both warped-nn classifiers perfectly separate both classes.

Experiment E5: In time series classification, it is common practice to normalize the data. In experiment E6, we applied a z-transformation on the time series from experiment E1. The euc-nn and twi-nn classifiers failed for the same reasons as in the previous experiment. The dtw-nn classifier failed, because the z-transformation eliminated the differences between constant segments such that both classes became indistinguishable. This experiment is one of the rare examples, where normalization deteriorates discriminative features

To summarize, there are cases for which the twi-distance is inappropriate and not competitive with the dtw-distance and vice versa. The twi-distance is inappropriate when constant segments are crucial for discriminating both classes but are outweighed by other factors. Conversely, the dtw-distance is inappropriate when constant segments are irrelevant and there differences are not outweighed by relevant factors. We could not detect either case in the UCR archive.

5 Conclusion

The dtw-distance fails to satisfy the triangle inequality and the identity of indiscernibles. As a consequence, the dtw-distance is not warping-invariant though otherwise stated in some publications. The lack of warping-invariance of the dtw-distance contributes to strange behavior and practical inconsistencies. For example, the strange behavior of the nearest-neighbor rule affects other time series mining methods such as k-means clustering. Subject to certain conditions, we can arbitrarily increase the cluster quality of k-means without changing the clusters. The practical inconsistency refers to the discrepancy between the underlying idea of the dtw-distance and how the dtw-distance is actually used in applications. The literature postulates that the dtw-distance can cope with temporal variations but stores and processes time series in a form as if the dtw-distance cannot cope with such variations. In doing so, the dtw nearest-neighbor classifier sacrifices storage-savings and computational efficiency without gaining substantially better error rates. The proposed twi-distance is a warping-invariant semi-metric that resolves both issues, the strange behavior of the dtw-distance and its practical inconsistency. The twi-distance requires less storage and is faster to compute than the dtw-distance without sacrificing for solution quality in nearest-neighbor classification for a broad range of problems. A limitation of the twi-distance are reducible time series whose constant segments are important discriminative features for classification. We could not detect such features among the $83$ datasets from the UCR archive. The proposed contribution challenges past use of the dtw-distance in nearest-neighbor classification and instead suggests the more efficient and theoretically more consistent twi-distance as default measure. Future research aims at studying to what extent the findings of this article carry over to time series mining techniques other than nearest-neighbor classification.

Acknowledgements.

B. Jain was funded by the DFG Sachbeihilfe JA 2109/4-2.

Appendix A Semi-Metrification of DTW-Spaces

This section converts the dtw-space to a semi-metric space and shows that the canonical extension of the proposed semi-metric is warping-invariant. The technical treatment is structured as follows: Section A.1 studies expansions, compressions, condensed forms, and irreducibility in terms of words over arbitrary alphabets. Sections A.2 and A.3 propose warping walks as a more general and convenient concept than warping paths. In addition, we study some properties of warping walks. Section A.4 constructs a semi-metric quotient space induced by the dtw-space as described in Section 2. Finally, Section A.5 proves that the canonical extension of the quotient metric is warping-invariant.

For the sake of readability, this section is self-contained and restates all definitions and notations mentioned in earlier sections. Throughout this section, we use the following notations:

Notation 9**.**

The set ${\mathbb{R}}_{\geq 0}$ is the set of non-negative reals, ${\mathbb{N}}$ is the set of positive integers, and ${\mathbb{N}}_{0}$ is the set of non-negative integers. We write $[n]=\mathop{\left\{1,\ldots,n\right\}}$ for $n\in{\mathbb{N}}$.

A.1 Prime Factorizations, Expansions, and Condensed Forms of Words

The key results of this article apply the same auxiliary results on warping paths and on time series. Both, warping paths and time series, can be regarded as words over different alphabets. Therefore, we derive the auxiliary results on the more general structure of words. We first propose the notions of prime words and prime factorization of words. Then we show that every word has a unique prime factorization. Using prime factorizations we define expansions, compressions, irreducible words, and condensed forms. Finally, we study the relationships between these different concepts.

Let ${\mathcal{A}}$ be a set, called alphabet. We impose no restrictions on the set ${\mathcal{A}}$. Thus, the elements of ${\mathcal{A}}$ can be symbols, strings, trees, graphs, functions, reals, feature vectors, and so on. A word over ${\mathcal{A}}$ is a finite sequence $x=x_{1}\cdots x_{n}$ with elements $x_{i}\in{\mathcal{A}}$ for all $i\in[n]$. The set ${\mathcal{A}}(x)=\mathop{\left\{x_{1},\ldots,x_{n}\right\}}$ is the set of elements contained in $x$. We denote by ${\mathcal{A}}^{*}$ the set of all words over ${\mathcal{A}}$ and by $\varepsilon$ the empty word. We write ${\mathcal{A}}^{+}={\mathcal{A}}\setminus\mathop{\left\{\varepsilon\right\}}$ to denote the set of non-empty words over ${\mathcal{A}}$.

Let $x=x_{1}\cdots x_{m}$ and $y=y_{1}\cdots y_{n}$ be two words over ${\mathcal{A}}$. The concatenation $z=xy$ is a word $z=z_{1}\cdots z_{m+n}$ with elements

[TABLE]

for all $i\in[m+n]$. The concatenation is an associative operation on ${\mathcal{A}}^{*}$. Hence, we can omit parentheses and write $xyz$ instead of $(xy)z$ or $x(yz)$ for all $x,y,z\in{\mathcal{A}}^{*}$. For a given $n\in{\mathbb{N}}_{0}$, we write $x^{n}$ to denote the $n$-fold concatenation of a word $x\in{\mathcal{A}}^{*}$. We set $x^{0}=\varepsilon$ for every $x\in{\mathcal{A}}^{*}$ and $\varepsilon^{n}=\varepsilon$ for all $n\in{\mathbb{N}}_{0}$.

The length of a word $x$ over ${\mathcal{A}}$, denoted by $\mathop{\left\lvert x\right\rvert}$, is the number of its elements. A prime word is a word of the form $x=a^{n}$, where $a\in{\mathcal{A}}$ and $n\in{\mathbb{N}}$ is a positive integer. From the definition follows that ${\mathcal{A}}(a^{n})=\mathop{\left\{a\right\}}$ and that the empty word $\varepsilon$ is not a prime word.

A non-empty word $v\in{\mathcal{A}}^{+}$ is a factor of a word $x\in{\mathcal{A}}^{*}$ if there are words $u,w\in{\mathcal{A}}^{*}$ such that $x=uvw$. Every non-empty word $x$ is a factor of itself, because $x=\varepsilon x\varepsilon$. In addition, the empty word $\varepsilon$ has no factors.

Let ${\mathcal{A}}^{**}=({\mathcal{A}}^{*})^{*}$ be the set of all finite words over the alphabet ${\mathcal{A}}^{*}$. A prime factorization of a word $x\in{\mathcal{A}}^{*}$ is a word $\Pi(x)=p_{1}\cdots p_{k}\in{\mathcal{A}}^{**}$ of prime factors $p_{t}\in{\mathcal{A}}^{*}$ of $x$ for all $t\in[k]$ such that

$p_{1}\cdots p_{k}=x$ (partition-condition) 2. 2.

${\mathcal{A}}(p_{t})\neq{\mathcal{A}}(p_{t+1})$ for all $t\in[k-1]$ (maximality-condition)

The partition-condition demands that the concatenation of all prime factors yields $x$. The maximality-condition demands that a prime factorization of $x$ consists of maximal prime factors. By definition, we have $\Pi(\varepsilon)=\varepsilon$.

Proposition 10**.**

Every word over ${\mathcal{A}}$ has a unique prime factorization.

Proof.

Since $\Pi(\varepsilon)=\varepsilon$, it is sufficient to consider non-empty words over ${\mathcal{A}}$. We first show that every $x\in{\mathcal{A}}^{+}$ has a prime factorization by induction over the length $\mathop{\left\lvert x\right\rvert}=n$.

Base case: Suppose that $n=1$. Let $x$ be a word over ${\mathcal{A}}$ of length one. Then $x$ is prime and a factor of itself. Hence, $\Pi(x)=x$ is a prime factorization of $x$.

Inductive step: Suppose that every word over ${\mathcal{A}}$ of length $n$ has a prime factorization. Let $x=x_{1}\cdots x_{n+1}$ be a word of length $\mathop{\left\lvert x\right\rvert}=n+1$. By induction hypothesis, the word $x^{\prime}=x_{1}\cdots x_{n}$ has a prime factorization $\Pi(x^{\prime})=p_{1}\cdots p_{k}$ for some $k\in[n]$. Suppose that $p_{k}=a^{n_{k}}$ for some $n_{k}\in{\mathbb{N}}$. Then ${\mathcal{A}}(p_{k})=a$. We distinguish between two cases:

Case $a=x_{n+1}$: The concatenation $\tilde{p}_{k}=p_{k}a=a^{n_{k}+1}$ is a prime word with ${\mathcal{A}}(\tilde{p}_{k})=\mathop{\left\{a\right\}}$. Let $q=p_{1}\cdots p_{k-1}$. We have

[TABLE]

Equation (1) shows the first property of a prime factorization and equation (2) shows that $\tilde{p}_{k}$ is a factor of $x$. From the induction hypothesis follows that ${\mathcal{A}}(p_{k-1})\neq{\mathcal{A}}(p_{k})={\mathcal{A}}(\tilde{p}_{k})$. This shows that $\Pi(x)=p_{1}\cdots p_{k-1}\tilde{p}_{k}$ is a prime factorization of $x$. 2. 2.

Case $a\neq x_{n+1}$: The word $p_{k+1}=x_{n+1}$ is a prime factor of $x$ satisfying $p_{1}\cdots p_{k}p_{k+1}=x$ and ${\mathcal{A}}(p_{k})=a\neq x_{n+1}={\mathcal{A}}(p_{k+1})$. Then from the induction hypothesis follows that $\Pi(x)=p_{1}\cdots p_{k+1}$ is a prime factorization of $x$.

It remains to show that a prime factorization is unique. Let $\Pi(x)=p_{1}\cdots p_{k}$ and $\Pi^{\prime}(x)=q_{1}\cdots q_{l}$ be different prime factorizations of $x$. Suppose that the prime factors are of the form $p_{r}=a_{r}^{m_{r}}$ and $q_{s}=b_{s}^{n_{s}}$ for all $r\in[k]$ and $s\in[l]$, where $a_{r},b_{s}\in{\mathcal{A}}$ and $m_{r},n_{s}\in{\mathbb{N}}$. The partition-condition of a prime factorization yields

[TABLE]

Since $\Pi(x)$ and $\Pi^{\prime}(x)$ are different, we can find a smallest index $i\in[k]\cap[l]$ such that $p_{i}\neq q_{i}$. From $p_{i}\neq q_{i}$ follows that $a_{i}^{m_{i}}\neq b_{i}^{n_{i}}$. Since $i$ is the smallest index for which the prime factors of both prime factorizations differ, we have

[TABLE]

There are words $v,w\in{\mathcal{A}}^{*}$ such that $x=ua_{i}v=ub_{i}w=x$. This shows that $a_{i}=b_{i}$. Hence, from $a_{i}^{m_{i}}\neq b_{i}^{n_{i}}$ and $a_{i}=b_{i}$ follows that $m_{i}\neq n_{i}$. We distinguish between two cases: (i) $m_{i}<n_{i}$ and (ii) $m_{i}>n_{i}$. We assume that $m_{i}<n_{i}$. Suppose that

[TABLE]

From $m_{i}<n_{i}$ follows $\mathop{\left\lvert u\right\rvert}<\mathop{\left\lvert x\right\rvert}$. This implies that $i<k$, that is $p_{i+1}$ is an element of $\Pi(x)$. Thus, there are words $v,w\in{\mathcal{A}}^{*}$ such that $x=ua_{i+1}v=ub_{i}w$. We obtain

[TABLE]

This violates the maximality-condition of a prime factorization and therefore contradicts our assumption that $m_{i}<n_{i}$. In a similar way, we can derive a contradiction for the second case $m_{i}>n_{i}$. Combining the results of both cases gives $m_{i}=n_{i}$, which contradicts our assumption that both prime factorizations are different. Thus a prime factorization is uniquely determined. This completes the proof. ∎

Let $x$ and $x^{\prime}$ be words over ${\mathcal{A}}$ whose prime factorizations $\Pi(x)=p_{1}\cdots p_{k}$ and $\Pi(y)=p^{\prime}_{1}\cdots p^{\prime}_{k}$, resp., have the same length $k$. We say $x$ is an expansion of $x^{\prime}$, written as $x\succ x^{\prime}$, if ${\mathcal{A}}(p_{t})={\mathcal{A}}(p^{\prime}_{t})$ and $\mathop{\left\lvert p_{t}\right\rvert}\geq\mathop{\left\lvert p^{\prime}_{t}\right\rvert}$ for all $t\in[k]$. If $x$ is an expansion of $x^{\prime}$, then $x^{\prime}$ is called a compression of $x$, denoted by $x^{\prime}\prec x$. By definition, every word is an expansion (compression) of itself. The set

[TABLE]

is the set of all compressions of $x$. Suppose that $x_{0},x_{1},\ldots,x_{k}$ are $k+1$ time series. Occasionally, we write $x_{0}\succ x_{1},\ldots,x_{k}$ for $x_{0}\succ x_{1},\ldots,x_{0}\succ x_{k}$ and similarly $x_{0}\prec x_{1},\ldots,x_{k}$ for $x_{0}\prec x_{1},\ldots,x_{0}\prec x_{k}$.

An irreducible word is a word $x=x_{1}\cdots x_{n}$ over ${\mathcal{A}}$ with prime factorization $\Pi(x)=x_{1}\cdots x_{n}$. From $\Pi(\varepsilon)=\varepsilon$ follows that the empty word $\varepsilon$ is irreducible. A condensed form of $x$ a word over ${\mathcal{A}}$ is an irreducible word $x^{*}$ such that $x\succ x^{*}$. The condensed form of $\varepsilon$ is $\varepsilon$. We show that every word has a uniquely determined condensed form.

Proposition 11**.**

Every word has a unique condensed form.

Proof.

It is sufficient to show existence and uniqueness of a condensed form of non-empty words. Let $x$ be a word over ${\mathcal{A}}$ with prime factorization $\Pi(x)=p_{1}\cdots p_{k}$. Let ${\mathcal{A}}(p_{t})=\mathop{\left\{a_{t}\right\}}$ for all $t\in[k]$. We define the word $x^{*}=a_{1}\cdots a_{k}$. Then $\Pi(x^{*})=a_{1}\cdots a_{k}$ is the prime factorization of $x^{*}$ that obviously satisfies the partition-condition and whose maximality-condition is inherited by the maximality-condition of the prime factorization $\Pi(x)$. This shows that $x^{*}$ is irreducible and a compression of $x$. Hence, $x^{*}$ is a condensed form of $x$.

Suppose that $z=b_{1}\cdots b_{l}$ is an irreducible word such that $z\prec x$. Then we have $k=l$, ${\mathcal{A}}(p_{t})={\mathcal{A}}(b_{t})$, and $\mathop{\left\lvert p_{t}\right\rvert}\geq\mathop{\left\lvert b_{t}\right\rvert}=1$ for all $t\in[k]$. Observe that $\mathop{\left\{a_{t}\right\}}={\mathcal{A}}(p_{t})={\mathcal{A}}(b_{t})=\mathop{\left\{b_{t}\right\}}$. We obtain $z=a_{1}\cdots a_{k}$ showing the uniqueness of the condensed form of $x$. ∎

The next result shows that expansions (compressions) are transitive.

Proposition 12**.**

Let $x,y,z\in{\mathcal{A}}^{*}$. From $x\succ y$ and $y\succ z$ follows $x\succ z$.

Proof.

Let $\Pi(x)=u_{1}\cdots u_{k}$, $\Pi(y)=v_{1}\cdots v_{l}$, and $\Pi(z)=w_{1}\cdots w_{m}$ be the prime factorizations of $x$, $y$, and $z$, respectively. From $x\succ y$ and $y\succ z$ follows that $k=l=m$. Let $t\in[k]$. We have $u_{t}\succ v_{t}$ and $v_{t}\succ w_{t}$. From $u_{t}\succ v_{t}$ follows that there are elements $a_{t}\in{\mathcal{A}}$ and $\alpha_{t},\beta_{t}\in{\mathbb{N}}$ with $\alpha_{t}\geq\beta_{t}$ such that $u_{t}=a_{t}^{\alpha_{t}}$ and $v_{t}=a_{t}^{\beta_{t}}$. From $v_{t}\succ w_{t}$ follows that there is an element $\gamma_{t}\in{\mathbb{N}}$ with $\beta_{t}\geq\gamma_{t}$ such that $w_{t}=a_{t}^{\gamma_{t}}$. Since $\alpha_{t}\geq\beta_{t}$ and $\beta_{t}\geq\gamma_{t}$, we obtain $u_{t}\succ w_{t}$. We have chosen $t\in[k]$ arbitrarily. Hence, we find that $x\succ z$. This proves the assertion. ∎

Expansions have been introduced as expansions on the prime factors. Lemma 13 states that expansions are obtained by replicating a subset of elements of a given word.

Lemma 13**.**

Let $x=x_{1}\cdots x_{n}$ and $y=y_{1}\cdots y_{m}$ be words over ${\mathcal{A}}$. Then the following statements are equivalent:

$x$* is an expansion of $y$.* 2. 2.

There are $\alpha_{1},\ldots,\alpha_{m}\in{\mathbb{N}}$ such that $x=y_{1}^{\alpha_{1}}\cdots y_{m}^{\alpha_{m}}$.

Proof.

Suppose that $\Pi(x)=p_{1}\cdots p_{k}$ and $\Pi(y)=q_{1}\cdots q_{l}$ are the prime factorization of $x$ and $y$, respectively.

$\Rightarrow$: We assume that $x$ is an expansion of $y$. Then from $x\succ y$ follows that $k=l$, ${\mathcal{A}}(p_{t})={\mathcal{A}}(q_{t})$, and $\mathop{\left\lvert p_{t}\right\rvert}\geq\mathop{\left\lvert q_{t}\right\rvert}$ for all $t\in[k]$. We arbitrarily pick an element $t\in[k]$. There are elements $a_{t}\in{\mathcal{A}}$ and $\beta_{t},\gamma_{t}\in{\mathbb{N}}$ such that $p_{t}=a_{t}^{\beta_{t}}$, $q_{t}=a_{t}^{\gamma_{t}}$ and $\beta_{t}\geq\gamma_{t}$. In addition, there is an index $i_{t}\in[m]$ such that

[TABLE]

Let $\nu_{t}=\beta_{t}-\gamma_{t}-1$. Then $\nu_{t}\geq 0$ and we have

[TABLE]

We set $\alpha_{i_{t}}=\nu_{t}$ and $\alpha_{i_{t}+1}=\cdots\alpha_{i_{t}+\gamma_{t}-1}=1$. Concatenating all prime factors $p_{t}$ of $x$ yields the assertion.

$\Leftarrow$: We assume that there are integers $\alpha_{1},\ldots,\alpha_{m}\in{\mathbb{N}}$ such that $x=y_{1}^{\alpha_{1}}\cdots y_{m}^{\alpha_{m}}$. Suppose that ${\mathcal{A}}(q_{t})=\mathop{\left\{a_{t}\right\}}$ for all $t\in[l]$. Then there are integers $i_{0},\ldots,i_{l}\in{\mathbb{N}}$ such that $0=i_{0}<i_{1}<\cdots<i_{l}=m$ and

[TABLE]

for all $t\in[l]$. We set $\beta_{t}=i_{t}-(i_{t-1}+1)$ and $\gamma_{t}=\alpha_{i_{t-1}+1}+\cdots+\alpha_{i_{t}}$ for all $t\in[l]$. From $1\leq\alpha_{i}$ for all $i\in[m]$ follows $\beta_{t}\leq\gamma_{t}$ for all $t\in[l]$. Hence, we have

[TABLE]

Hence, $k=l$, ${\mathcal{A}}(p_{t})={\mathcal{A}}(q_{t})$, and $\mathop{\left\lvert p_{t}\right\rvert}=\gamma_{t}\geq\beta_{t}=\mathop{\left\lvert q_{t}\right\rvert}$ for all $t\in[l]$. This shows $x\succ y$. ∎

The next result shows that the set of compressions of an irreducible word is a singleton.

Proposition 14**.**

Let $x\in{\mathcal{A}}^{*}$ be irreducible. Then ${\mathcal{C}}(x)=\mathop{\left\{x\right\}}$.

Proof.

By definition, we have $x\in{\mathcal{C}}(x)$. Suppose there is a word $y\in{\mathcal{C}}(x)$ with prime factorization $\Pi(y)=p_{1}\cdots p_{k}$. Let $y^{*}=a_{1}\cdots a_{k}$ be the condensed form of $y$, where $a_{t}\in{\mathcal{A}}(p_{t})$ for all $t\in[k]$. From $y\prec x$ and $y^{*}\prec y$ follows $y^{*}\prec x$ by Prop. 12. According to Lemma 13 there are positive integers $\alpha_{1},\ldots,\alpha_{k}\in{\mathbb{N}}$ such that $x=a_{1}^{\alpha_{1}}\cdots a_{k}^{\alpha_{k}}$. Since $x$ is irreducible all $\alpha_{t}$ have value one giving $x=a_{1}\cdots a_{k}$. Hence, we have $x=y^{*}$. In addition, from $x\succ y$ and $y\succ x$ follows $x=y$. This shows the assertion. ∎

Proposition 15 states that every compression of a word is an expansion of its condensed form.

Proposition 15**.**

Let $x$ be a word over ${\mathcal{A}}$ with condensed form $x^{*}$. Suppose that $y\in{\mathcal{A}}^{*}$ such that $y\prec x$. Then $x^{*}\prec y$.

Proof.

Let $x\in{\mathcal{A}}^{*}$ be a word with prime factorization $\Pi(x)=p_{1}\cdots p_{k}$. Suppose that $y\in{\mathcal{C}}(x)$ with condensed form $\Pi(y)=q_{1}\cdots q_{l}$. From $y\prec x$ follows that $k=l$ and $q_{t}\prec p_{t}$ for all $t\in[k]$. This implies that $x$ and $y$ have the same condensed form $x^{*}$. Hence, we have $x^{*}\prec y$, which completes the proof. ∎

Suppose that ${\mathcal{C}}(x)$ is the set of compressions of a word $x$. We show that the shortest word in ${\mathcal{C}}(x)$ is the condensed form of $x$.

Proposition 16**.**

Let $x$ be a word over ${\mathcal{A}}$ with condensed form $x^{*}$. Then $\mathop{\left\lvert x^{*}\right\rvert}<\mathop{\left\lvert y\right\rvert}$ for all $y\in{\mathcal{C}}(x)\setminus\mathop{\left\{x^{*}\right\}}$.

Proof.

Let $x^{*}=a_{1}\cdots a_{k}$ and let $y\in{\mathcal{C}}(x)$. From Prop. 15 follows that $x^{*}\prec y$. Lemma 13 gives positive integers $\alpha_{1},\ldots\alpha_{k}\in{\mathbb{N}}$ such that $y=a_{1}^{\alpha_{1}}\cdots a_{k}^{\alpha_{k}}$. This shows that $\mathop{\left\lvert y\right\rvert}=\alpha_{1}+\cdots+\alpha_{k}\geq k=\mathop{\left\lvert x^{*}\right\rvert}$. Suppose that $\mathop{\left\lvert y\right\rvert}=k$. In this case, we have $\alpha_{1}=\cdots=\alpha_{k}=1$ and therefore $y=x^{*}$. This shows that $\mathop{\left\lvert x^{*}\right\rvert}<\mathop{\left\lvert y\right\rvert}$ for all $y\in{\mathcal{C}}(x)\setminus\mathop{\left\{x^{*}\right\}}$. ∎

Suppose that $x,y,z$ are words over ${\mathcal{A}}^{*}$. We say, $z$ is a common compression of $x$ and $y$ if $z\prec x,y$. A compression-expansion (co-ex) function is a function $f:{\mathcal{A}}^{*}\rightarrow{\mathcal{A}}^{*}$ such that there is a common compression of $x$ and $f(x)$. Proposition 17 states that co-ex functions are closed under compositions.

Proposition 17**.**

The composition of two co-ex functions is a co-ex function.

Proof.

To prove the assertion, we repeatedly apply transitivity of expansions (Prop. 12). Let $x\in{\mathcal{A}}^{*}$ and let $f=g\circ h$ be the composition of two co-ex functions $g,h:{\mathcal{A}}^{*}\rightarrow{\mathcal{A}}^{*}$. Then there are words $z_{h}$ and $z_{g}$ such that $z_{h}\prec x,h(x)$ and $z_{g}\prec h(x),g(h(x))$. Suppose that $z_{h}^{*}$ and $z_{g}^{*}$ are the condensed forms of $z_{h}$ and $z_{g}$, respectively. From

[TABLE]

follows $z_{h}^{*}\prec h(x)$ and $z_{g}^{*}\prec h(x)$ by the transitivity of expansions. According to Prop. 11, the condensed form of a word is unique. Hence, we have $z_{h}^{*}=z_{g}^{*}$. We set $x^{*}=z_{h}^{*}=z_{g}^{*}$. Then from

[TABLE]

follows $x^{*}\prec x,f(x)$ by the transitivity of expansions. This shows that $x^{*}$ is a common compression of $x$ and $f(x)$. Since $x$ was chosen arbitrarily, the assertion follows. ∎

A.2 Warping Walks

The standard definition of the dtw-distance is inconvenient for our purposes. The recursive definition of warping paths is easy to understand and well-suited for deriving algorithmic solutions, but often less suited for a theoretical analysis. In addition, warping paths are not closed under compositions. As a more convenient definition, we introduce warping walks. Warping walks generalize warping paths by slightly relaxing the step condition. Using warping functions and matrices, this section shows that warping walks do not affect the dtw-distance. The next section shows that warping walks are closed under compositions.

Notation 18**.**

Let ${\mathbb{B}}=\mathop{\left\{0,1\right\}}$ and let $I_{n}\in{\mathbb{R}}^{n\times n}$ be the identity matrix. $\square$

Let $\ell,n\in{\mathbb{N}}$. A function $\phi:[\ell]\rightarrow[n]$ is a warping function if it is surjective and monotonically increasing. Thus, for a warping function we always have $\ell\geq n$. The warping matrix associated with warping function $\phi$ is a matrix of the form

[TABLE]

where $e_{i}$ is the $i$-th standard basis vector of ${\mathbb{R}}^{n}$, denoted as a row vector, with $1$ in the $i$-th position and [math] in every other position. The next result shows the effect of multiplying a time series with a warping matrix.

Lemma 19**.**

Let $\phi:[\ell]\rightarrow[n]$ be a warping function with associated warping matrix $\Phi$. Suppose that $x=(x_{1},\ldots,x_{n})s\in{\mathcal{T}}$ is a time series of length $\mathop{\left\lvert x\right\rvert}=n$. Then there are elements $\alpha_{1},\ldots,\alpha_{n}\in{\mathbb{N}}$ such that

[TABLE]

Proof.

Since $\phi$ is surjective and monotonic, we can find integers $\alpha_{1},\ldots,\alpha_{n}\in{\mathbb{N}}$ such that

[TABLE]

for all $i\in[\ell]$. Let $\Phi\in{\mathbb{B}}^{\ell\times n}$ be the warping matrix associated with $\phi$. Then the $n$ rows $\Phi_{i}$ of $\Phi$ are of the form

[TABLE]

Obviously, the warping matrix $\Phi$ satisfies

[TABLE]

∎

A warping walk is a pair $w=(\phi,\psi)$ consisting of warping functions $\phi:[\ell]\rightarrow[m]$ and $\psi:[\ell]\rightarrow[n]$ of the same domain $[l]$. The warping walk $w$ has order $m\times n$ and length $\ell$. By ${\mathcal{W}}_{m,n}$ we denote the set of all warping walks of order $m\times n$ and of finite length.

In the classical terminology of dynamic time warping, a warping walk can be equivalently expressed by a sequence $w=(w_{1},\ldots,w_{\ell})$ of $\ell$ points $w_{l}=(\phi(l),\psi(l))\in[m]\times[n]$ such that the following conditions are satisfied:

$w_{1}=(1,1)$ and $w_{\ell}=(m,n)$ (boundary condition) 2. 2.

$w_{l+1}-w_{l}\in{\mathbb{B}}\times{\mathbb{B}}$ for all $l\in[\ell-1]$ (weak step condition)

The weak step condition relaxes the standard step condition of warping paths by additionally allowing zero-steps of the form $w_{l}-w_{l+1}=(0,0)$. Zero-steps duplicate points $w_{l}$ and thereby admit multiple correspondences between the same elements of the underlying time series.

Notation 20**.**

We identify warping walks $(\phi,\psi)$ with their associated warping matrices $(\Phi,\Psi)$ and interchangeably write $(\phi,\psi)\in{\mathcal{W}}_{m,n}$ and $(\Phi,\Psi)\in{\mathcal{W}}_{m,n}$.

A warping walk $p=(p_{1},\ldots,p_{\ell})$ is a warping path if $p_{l+1}\neq p_{l}$ for all $l\in[\ell-1]$. By ${\mathcal{P}}_{m,n}$ we denote the subset of all warping paths of order $m\times n$. Thus, a warping path is a warping walk without consecutive duplicates. Equivalently, a warping path satisfies the boundary condition and the strict step condition

1. $2^{\prime}.$

$w_{l+1}-w_{l}\in{\mathbb{B}}\times{\mathbb{B}}\setminus\mathop{\left\{0,0\right\}}$ for all $l\in[\ell-1]$ (strict step condition)

Warping walks are words over the alphabet ${\mathcal{A}}={\mathbb{N}}\times{\mathbb{N}}$ and warping paths are irreducible words over ${\mathcal{A}}$. For the sake of convenience, we regard ${\mathcal{W}}_{m,n}$ and ${\mathcal{P}}_{m,n}$ as subsets of ${\mathcal{A}}^{*}$. The condensation map

[TABLE]

sends a word $w$ over ${\mathcal{A}}$ to its condensed form $w^{*}$.

Lemma 21**.**

Let ${\mathcal{A}}={\mathbb{N}}\times{\mathbb{N}}$ and let $c:{\mathcal{A}}^{*}\rightarrow{\mathcal{A}}^{*}$ be the condensation map. Then $c({\mathcal{W}}_{m,n})={\mathcal{P}}_{m,n}$ for all $m,n\in{\mathbb{N}}$.

Proof.

Let $m,n\in{\mathbb{N}}$.

${\mathcal{P}}_{m,n}\subseteq c({\mathcal{W}}_{m,n})$: Let $p=(p_{1},\ldots,p_{\ell})\in{\mathcal{P}}_{m,n}$ be a warping path. From the strict step condition follows that $p$ is irreducible. Consider the word $w=(p_{1},\ldots,p_{\ell},p_{\ell+1})$, where $p_{\ell}=p_{\ell+1}$. The word $w$ satisfies the boundary and weak step condition. Hence, $w$ is a warping walk with unique prime factorization $\Pi(w)=(p_{1}\cdots p_{\ell})$. This shows that $p$ is the unique condensed form of $w$. Hence, we have ${\mathcal{P}}_{m,n}\subseteq c({\mathcal{W}}_{m,n})$.

$c({\mathcal{W}}_{m,n})\subseteq{\mathcal{P}}_{m,n}$: A warping walk $w\in{\mathcal{W}}_{m,n}$ satisfies the boundary and the weak step condition. As an irreducible word, the condensed form $w^{*}=c(w)$ satisfies the boundary and the strict step condition. Hence, $w^{*}$ is a warping path. This proves $c({\mathcal{W}}_{m,n})\subseteq{\mathcal{P}}_{m,n}$. ∎

The dtw-distance is a distance function on ${\mathcal{T}}$ of the form

[TABLE]

From [33], Prop. A.2 follows that the dtw-distance coincides with the standard definition of the dtw-distance as presented in Section 2. The next result expresses the dtw-distance in terms of warping walks.

Proposition 22**.**

Let $x,y\in{\mathcal{T}}$ be two time series. Then we have

[TABLE]

Proof.

Let $w\in{\mathcal{W}}_{m,n}$ be a warping walk. Then $p=c(w)$ is a warping path and a condensed form of $w$ by Lemma 21. From Prop. 16 follows that $\mathop{\left\lvert p\right\rvert}\leq\mathop{\left\lvert w\right\rvert}$. Thus, we obtain

[TABLE]

This implies

[TABLE]

and proves the assertion. ∎

We call a warping walk $(\Phi,\Psi)$ optimal if $\mathop{\left\lVert\Phi x-\Psi y\right\rVert}=\operatorname{\delta}(x,y)$. From Prop. 22 follows that transition from warping paths to warping walks leaves the dtw-distance unaltered and that we can condense every optimal warping walk to an optimal warping path by removing consecutive duplicates.

A.3 Properties of Warping Functions

In this section, we compile results on compositions of warping functions and warping walks. We begin with showing that warping functions are closed under compositions.

Lemma 23**.**

Let $\phi:[\ell]\rightarrow[m]$ and $\psi:[m]\rightarrow[n]$ be warping functions. Then the composition

[TABLE]

is also a warping function.

Proof.

The assertion follows, because the composition of surjective (monotonic) functions is surjective (monotonic). ∎

The composition of warping functions is contravariant to the composition of their associated warping matrices. Suppose that $\Phi\in{\mathbb{B}}^{\ell\times m}$ and $\Psi\in{\mathbb{B}}^{m\times n}$ are the warping matrices of the warping functions $\phi$ and $\psi$ from Lemma 23, respectively. Then the warping matrix of the composition $\theta=\psi\circ\phi$ is the matrix product $\Phi\Psi\in{\mathbb{B}}^{\ell\times n}$. The next result shows that warping walks are closed for a special form of compositions.

Lemma 24**.**

Let $(\phi,\psi)\in{\mathcal{W}}_{m,n}$ be a warping walk and let $\theta:[m]\rightarrow[r]$ be a warping function. Then $(\theta\circ\phi,\psi)$ is a warping walk in ${\mathcal{W}}_{r,n}$.

Proof.

Follows from Lemma 23 and by definition of a warping walk. ∎

Let $\phi:[m]\rightarrow[n]$ and $\phi^{\prime}:[m^{\prime}]\rightarrow[n]$ be warping functions. The pullback of $\phi$ and $\phi^{\prime}$ is the set of the form

[TABLE]

By $\pi:\phi\otimes\phi^{\prime}\rightarrow[m]$ and $\pi^{\prime}:\phi\otimes\phi^{\prime}\rightarrow[m^{\prime}]$ we denote the canonical projections. Let $\psi=\phi\circ\pi$ and $\psi^{\prime}=\phi^{\prime}\circ\pi^{\prime}$ be the compositions that send elements from the pullback $\phi\otimes\phi^{\prime}$ to the set $[n]$. The fiber of $i\in[n]$ under the map $\psi$ is the set ${\mathcal{F}}(i)=\mathop{\left\{(u,u^{\prime})\in\phi\otimes\phi^{\prime}\,:\,\psi(u,u^{\prime})=i\right\}}$. In a similar way, we can define the fiber of $i$ under the map $\psi^{\prime}$. The next results show some properties of pullbacks and their fibers.

Lemma 25**.**

Let $\phi\otimes\phi^{\prime}$ be a pullback of warping functions $\phi$ and $\phi^{\prime}$. Then the compositions $\psi=\phi\circ\pi$ and $\psi^{\prime}=\phi^{\prime}\circ\pi^{\prime}$ are surjective and satisfy $\psi=\psi^{\prime}$.

Proof.

Warping functions and the natural projections are surjective. As a composition of surjective functions, the functions $\psi$ and $\psi^{\prime}$ are surjective. For every $(u,u^{\prime})\in\phi\otimes\phi^{\prime}$ we have

[TABLE]

This proves the assertion $\psi=\psi^{\prime}$. ∎

Lemma 25 has the following implications: First, from $\psi=\psi^{\prime}$ follows that the fiber of $i$ under the map $\psi$ coincides with the fiber of $i$ under the map $\psi^{\prime}$. Second, since $\psi$ is surjective, the fibers ${\mathcal{F}}(i)$ are non-empty for every $i\in[n]$. Third, the fibers ${\mathcal{F}}(i)$ form a partition of the pullback $\phi\otimes\phi^{\prime}$.

Lemma 26**.**

Let $\phi\otimes\phi^{\prime}$ be a pullback of warping functions $\phi:[m]\rightarrow[n]$ and $\phi^{\prime}:[m^{\prime}]\rightarrow[n]$. Suppose that $i,j\in[n]$ with $i<j$. From $(u_{i},u^{\prime}_{i})\in{\mathcal{F}}(i)$ and $(u_{j},u^{\prime}_{j})\in{\mathcal{F}}(j)$ follows $u_{i}\leq u_{j}$ and $u^{\prime}_{i}\leq u^{\prime}_{j}$.

Proof.

We show the first assertion $u_{i}\leq u_{j}$. The proof for the second assertion $u^{\prime}_{i}\leq u^{\prime}_{j}$ is analogous. Suppose that $u_{i}>u_{j}$. From $(u_{i},u^{\prime}_{i})\in{\mathcal{F}}(i)$ follows $\phi(u_{i})=i$ and from $(u_{j},u^{\prime}_{j})\in{\mathcal{F}}(j)$ follows $\phi(u_{j})=j$. Since $\phi$ is monotonic, we have $i=\phi(u_{i})\geq\phi(u_{j})=j$, which contradicts the assumption that $i<j$. This shows $u_{i}\leq u_{j}$. ∎

Lemma 27**.**

Let $\phi\otimes\phi^{\prime}$ be a pullback of warping functions $\phi:[m]\rightarrow[n]$ and $\phi^{\prime}:[m^{\prime}]\rightarrow[n]$. For every $i\in[n]$ there are elements $u_{i}\in[m]$, $u^{\prime}_{i}\in[m^{\prime}]$ and $k_{i},l_{i}\in{\mathbb{N}}$ such that

[TABLE]

Proof.

We show the assertion for $\pi({\mathcal{F}}(i))$. The proof of the assertion for $\pi^{\prime}({\mathcal{F}}(i))$ is analogous. Let $i\in[n]$ and let ${\mathcal{G}}(i)=\pi({\mathcal{F}}(i))$. Since fibers are non-empty and finite, we can find elements $u_{i}\in[m]$ and $k_{i}\in{\mathbb{N}}$ such that ${\mathcal{G}}(i)=\mathop{\left\{u_{i},u_{i+1},\ldots,u_{i+k_{i}}\right\}}$ with $u_{i}<u_{i+1}<\cdots<u_{i+k_{i}}$. It remains to show that $u_{i+r}=u_{i}+r$ for all $r\in[k_{i}]$.

We assume that there is a smallest number $r\in[k_{i}]$ such that $u_{i+r}\neq u_{i}+r$. Then $r\geq 1$ and therefore $i+r-1\geq i$. This shows that $u_{i+r-1}\in{\mathcal{G}}(i)$. Observe that $u_{i+r-1}=u_{i}+r-1$, because $r$ is the smallest number violating $u_{i+r}=u_{i}+r$. From $u_{i+r-1}<u_{i+r}$ together with $u_{i}+r\notin{\mathcal{G}}(i)$ follows

[TABLE]

Recall that the fibers form a partition of the pullback $\phi\otimes\phi^{\prime}$. Then there is a $j\in[n]\setminus\mathop{\left\{i\right\}}$ such that $u_{i}+r\in{\mathcal{G}}(j)$. We distinguish between two cases:333The case $i=j$ can not occur by assumption.

Case $i<j$: From Lemma 26 follows that $u_{i+r}\leq u_{i}+r$, which contradicts the previously derived inequality $u_{i}+r<u_{i+r}$.

Case $j<i$: From Lemma 26 follows that $u_{i}+r\leq u_{i}$. Observe that either $u_{i}=u_{i+r-1}$ or $u_{i}<u_{i+r-1}$. We obtain the contradiction $u_{i}\leq u_{i+r-1}<u_{i}+r\leq u_{i}$.

From both contradictions follows that $u_{i+r}=u_{i}+r$ for every $r\in[k_{i}]$. This completes the proof. ∎

Lemma 28 uses pullbacks to show that pairs of warping functions with the same co-domain can be equalized by composition with a suitable warping walk.

Lemma 28**.**

Let $\phi:[m]\rightarrow[n]$ and $\phi^{\prime}:[m^{\prime}]\rightarrow[n]$ be warping functions. Then there are warping functions $\theta:[\ell]\rightarrow[m]$ and $\theta^{\prime}:[\ell]\rightarrow[m^{\prime}]$ for some $\ell\geq\max(m,m^{\prime})$ such that $\phi\circ\theta=\phi^{\prime}\circ\theta^{\prime}$.

Proof.

We first construct a suitable set ${\mathcal{Z}}$ of cardinality $\mathop{\left\lvert Z\right\rvert}=\ell$. For this, let $\phi\otimes\phi^{\prime}$ be the pullback of $\phi$ and $\phi^{\prime}$ and let $i\in[n]$. From Lemma 27 follows that there are elements $u_{i}\in[m]$, $u^{\prime}_{i}\in[m^{\prime}]$ and $k_{i},l_{i}\in{\mathbb{N}}$ such that

[TABLE]

where $u_{i+r}=u_{i}+r$ for all $r\in[k_{i}]$ and $u^{\prime}_{i+s}=u^{\prime}_{i}+s$ for all $s\in[l_{i}]$. Without loss of generality we assume that $k_{i}\leq l_{i}$. For every $i\in[n]$, we construct the subset

[TABLE]

and form their disjoint union

[TABLE]

Let $\leq_{{\mathcal{Z}}}$ be the lexicographical order on ${\mathcal{Z}}$ defined by

[TABLE]

for all $(u_{i},u^{\prime}_{i}),(u_{j},u^{\prime}_{j})\in{\mathcal{Z}}$. We show that the properties of ${\mathcal{Z}}$ yield a tighter condition on $\leq_{{\mathcal{Z}}}$. Let $(u_{i},u^{\prime}_{i}),(u_{j},u^{\prime}_{j})\in{\mathcal{Z}}$ such that $(u_{i},u^{\prime}_{i})\leq_{{\mathcal{Z}}}(u_{j},u^{\prime}_{j})$. Then there are $i,j\in[n]$ such that $(u_{i},u^{\prime}_{i})\in{\mathcal{Z}}(i)$ and $(u_{j},u^{\prime}_{j})\in{\mathcal{Z}}(j)$. We distinguish between three cases:

$i=j$: By construction of ${\mathcal{Z}}(i)$, the relationship $u_{i}\leq u_{j}$ gives $u^{\prime}_{i}\leq u^{\prime}_{j}$. 2. 2.

$i<j$: From Lemma 26 follows that $u_{i}\leq u_{j}$ and $u^{\prime}_{i}\leq u^{\prime}_{j}$. 3. 3.

$i>j$: Lemma 26 yields $u_{i}\geq u_{j}$ and $u^{\prime}_{i}\geq u^{\prime}_{j}$. The assumption $(u_{i},u^{\prime}_{i})\leq_{{\mathcal{Z}}}(u_{j},u^{\prime}_{j})$ gives $u_{i}\leq u_{j}$. Then from $u_{i}\geq u_{j}$ and $u_{i}\leq u_{j}$ follows $u_{i}=u_{j}$. In addition, we have $u^{\prime}_{i}\leq u^{\prime}_{j}$ by $(u_{i},u^{\prime}_{i})\leq_{{\mathcal{Z}}}(u_{j},u^{\prime}_{j})$ and $u_{i}=u_{j}$. Hence, from $u^{\prime}_{i}\geq u^{\prime}_{j}$ and $u^{\prime}_{i}\leq u^{\prime}_{j}$ follows $u^{\prime}_{i}=u^{\prime}_{j}$. In summary, we have $u_{i}=u_{j}$ and $u^{\prime}_{i}=u^{\prime}_{j}$.

The case distinction yields

[TABLE]

for all $(u_{i},u^{\prime}_{i}),(u_{j},u^{\prime}_{j})\in{\mathcal{Z}}$. Suppose that $\ell=\mathop{\left\lvert{\mathcal{Z}}\right\rvert}$. Then there is a bijective function

[TABLE]

such that $i\leq j$ implies $f(i)\leq_{{\mathcal{Z}}}f(j)$ for all $i,j\in[\ell]$.

Next, we show that the functions $\theta=\pi\circ f$ and $\theta^{\prime}=\pi^{\prime}\circ f$ are warping functions on $[\ell]$. Both functions $\theta$ and $\theta^{\prime}$ are surjective as compositions of surjective functions. To show that $\theta$ and $\theta^{\prime}$ are monotonic, we assume that $i,j\in[\ell]$ with $i<j$. Suppose that $f(i)=(u_{i},u^{\prime}_{i})$ and $f(j)=(u_{j},u^{\prime}_{j})$. From $i<j$ follows $(u_{i},u^{\prime}_{i})\leq_{{\mathcal{Z}}}(u_{j},u^{\prime}_{j})$ by construction of $f$. From $(u_{i},u^{\prime}_{i})\leq_{{\mathcal{Z}}}(u_{j},u^{\prime}_{j})$ follows $u_{i}\leq u_{j}$ and $u^{\prime}_{i}\leq u^{\prime}_{j}$ as shown in the first part of this proof. Hence, we find that

[TABLE]

Thus, $\theta$ and $\theta^{\prime}$ are monotonic. This proves that $\theta$ and $\theta^{\prime}$ are warping functions.

It remains to show $\phi\circ\theta=\phi^{\prime}\circ\theta^{\prime}$. From Lemma 25 follows $\phi\circ\pi=\phi^{\prime}\circ\pi^{\prime}$. Then we have

[TABLE]

This completes the proof. ∎

A.4 Semi-Metrification of DTW-Spaces

In this section, we convert the dtw-distance to a semi-metric. For this, we regard time series as words over the alphabet ${\mathcal{A}}={\mathbb{R}}$. The set of finite time series is denoted by ${\mathcal{T}}={\mathcal{A}}^{*}$. The next result shows that expansions of words over numbers can be expressed by matrix multiplication.

Lemma 29**.**

Let $x,y\in{\mathcal{T}}$ be two time series. Then the following statements are equivalent:

$x$* is an expansion of $y$.* 2. 2.

There is a warping matrix $\Phi$ such that $x=\Phi y$.

Proof.

Suppose that $\mathop{\left\lvert x\right\rvert}=n$ and $y=(y_{1},\ldots,y_{m})$.

$\Rightarrow$: We assume that $x\succ y$. According to Lemma 13 there are positive integers $\alpha_{1},\ldots,\alpha_{m}\in{\mathbb{N}}$ such that $n=\alpha_{1}+\cdots+\alpha_{m}$ and $x=y_{1}^{\alpha_{1}}\cdots y_{m}^{\alpha_{m}}$. Consider the function $\phi:[n]\rightarrow[m]$ defined by

[TABLE]

for all $i\in[n]$. The function $\phi$ is surjective: Suppose that $j\in[m]$. We set $i=\alpha_{1}+\cdots+\alpha_{j}$. Then $1\leq i\leq n$ and $\phi(i)=j$ by definition of $\phi$. By construction, the function $\phi$ is also monotonically increasing. Hence, $\phi$ is a warping function. Then from Lemma 19 follows the second statement.

$\Leftarrow$: Let $\Phi\in{\mathbb{B}}^{n\times m}$ be a warping matrix such that $x=\Phi y$. Then there is a warping function $\phi:[n]\rightarrow[m]$ associated with $\Phi$. The first statement follows by first applying Lemma 19 and then by Lemma 13. ∎

Warping identification is a relation on ${\mathcal{T}}$ defined by $x\sim y\,\Leftrightarrow\,\operatorname{\delta}(x,y)=0$ for all $x,y\in{\mathcal{T}}$. We show that warping identification is an equivalence relation as claimed in Prop. 2.

Section 2.4, Prop. 2.

The warping-identification $\sim$ is an equivalence relation on ${\mathcal{T}}$.

Proof.

The relation $\sim$ is reflexive and symmetric by the properties of the dtw-distance. It remains to show that the warping-identification is transitive. Let $x,y,z\in{\mathcal{T}}$ be time series with $x\sim y$ and $y\sim z$. Then $\operatorname{\delta}(x,y)=\operatorname{\delta}(y,z)=0$. Hence, there are optimal warping paths $(\Phi,\Psi)$ and $(\Phi^{\prime},\Psi^{\prime})$ of length $\ell$ and $\ell^{\prime}$, resp., such that $\mathop{\left\lVert\Phi x-\Psi y\right\rVert}=\mathop{\left\lVert\Phi^{\prime}y-\Psi^{\prime}z\right\rVert}=0$. From Lemma 28 follows that there are warping matrices $\Theta$ and $\Theta^{\prime}$ of the same length $\ell$ such that $\Theta\Psi y=\Theta^{\prime}\Psi^{\prime}y$. Observe that

[TABLE]

Note that the second inequality in the third line follows from the fact that the Frobenius norm on matrices is compatible to the vector norm. From Lemma 23 follows that the products $\Theta\Phi$ and $\Theta^{\prime}\Psi^{\prime}$ are warping matrices. Since both products have the same length $\ell$, we find that the pair $(\Theta\Phi,\Theta^{\prime}\Psi^{\prime})$ is a warping walk. Then from Prop. 22 follows that $\operatorname{\delta}(x,z)\leq\mathop{\left\lVert\Theta\Phi x-\Theta^{\prime}\Psi^{\prime}z\right\rVert}=0$. This proves that $\sim$ is transitive and completes the proof. ∎

For every $x\in{\mathcal{T}}$ let $[x]=\mathop{\left\{y\in{\mathcal{T}}\,:\,x\sim y\right\}}$ denote the warping-identification class of $x$. The next result presents an equivalent definition of the warping-identification class.

Section 2.4, Prop. 3

. Let $x\in{\mathcal{T}}$ be a time series with condensed form $x^{*}$. Then the warping-identification class of $x$ is of the form

[TABLE]

Proof.

The warping-identification class is defined by

[TABLE]

Let ${\mathcal{E}}(x^{*})=\mathop{\left\{y\in{\mathcal{T}}\,:\,y\succ x^{*}\right\}}$ denote the set of expansions of $x^{*}$. We show that $[x]={\mathcal{E}}(x^{*})$.

$\subseteq$: Let $y\in[x]$. There is an optimal warping path $(\Phi,\Psi)$ such that $\operatorname{\delta}(x,y)=\mathop{\left\lVert\Phi x-\Psi y\right\rVert}=0$. From Lemma 29 follows that there is a warping matrix $\Theta$ with $x=\Theta x^{*}$. Hence, $\mathop{\left\lVert\Phi\Theta x^{*}-\Psi y\right\rVert}=0$ and we obtain $\Phi\Theta x^{*}=\Psi y$. From Lemma 23 follows that the product $\Phi\Theta$ of warping matrices $\Phi$ and $\Theta$ is a warping matrix. We set $z=\Phi\Theta x^{*}=\Psi y$. Then $z\succ x^{*}$ and $z\succ y$ by Lemma 29. From Prop. 15 follows that $y\succ x^{*}$. This shows that $y\in{\mathcal{E}}(x^{*})$.

$\supseteq$: Let $y\in{\mathcal{E}}(x^{*})$. We assume that $\mathop{\left\lvert x\right\rvert}=n$, $\mathop{\left\lvert y\right\rvert}=m$, and $\mathop{\left\lvert x^{*}\right\rvert}=k$. From Lemma 29 follows that there are warping matrices $\Phi\in{\mathbb{B}}^{n\times k}$ and $\Psi\in{\mathbb{B}}^{m\times k}$ with $x=\Phi x^{*}$ and $y=\Psi x^{*}$, respectively. Then $(\Phi,I_{n})$ and $(\Psi,I_{m})$ are warping walks of length $n$ and $m$ respectively. We have

[TABLE]

giving $\operatorname{\delta}(x,x^{*})=\operatorname{\delta}(y,x^{*})=0$. Hence, we have $x\sim x^{*}$ and $y\sim x^{*}$. Since $\sim$ is an equivalence relation, we have $x\sim y$ by Prop. 2. This proves $y\in[x]$. ∎

We prove Prop. 4 from Section 2.4.

Section 2.4, Prop. 4.

Let $x,y\in{\mathcal{T}}$ be time series such that $x\succ y$. Then

$\operatorname{\delta}(x,y)=0$ 2. 2.

$\operatorname{\delta}(x,z)\geq\operatorname{\delta}(y,z)$ for all $z\in{\mathcal{T}}$.

Proof.

We first show the second assertion. Let $z\in{\mathcal{T}}$ be a time series and let $(\Phi,\Psi)$ be an optimal warping path between $x$ and $z$. Then we have $\delta(x,z)=\mathop{\left\lVert\Phi x-\Psi z\right\rVert}$ by Prop. 22. From $x\succ y$ and Lemma 29 follows that there is a warping matrix $\Theta$ such that $x=\Theta y$. We obtain

[TABLE]

From Lemma 24 follows that $(\Phi\Theta,\Psi)$ is a warping walk. The inequality holds, because $(\Phi\Theta,\Psi)$ is not necessarily an optimal warping walk between $y$ and $z$.

The first assertion follows from the second one by setting $z=x$. We obtain

[TABLE]

This implies $\operatorname{\delta}(y,x)=0$ and completes the proof. ∎

The set ${\mathcal{T}}^{*}=\mathop{\left\{[x]\,:\,x\in{\mathcal{T}}\right\}}$ is the quotient space of ${\mathcal{T}}$ under warping identification $\sim$. We define the distance function

[TABLE]

We call $\delta^{*}$ the time-warp-invariant (twi) distance induced by $\delta$.

Section 2.4, Theorem 5.

The twi-distance $\delta^{*}$ induced by the dtw-distance $\delta$ is a well-defined semi-metric satisfying $\delta^{*}([x],[y])=\delta(x^{*},y^{*})$ for all $x,y\in{\mathcal{T}}$.

Proof.

Let $x^{*}$ and $y^{*}$ be the condensed forms of $x$ and $y$, respectively. Repeatedly applying Prop. 4 gives

[TABLE]

for all $x^{\prime}\in[x]$ and all $y^{\prime}\in[y]$. Hence, the infimum exists and $\operatorname{\delta}^{*}([x],[y])=\operatorname{\delta}(x^{*},y^{*})$ is well-defined.

We show that $\delta^{*}$ is a semi-metric. Non-negativity and symmetry of $\operatorname{\delta}^{*}$ follow from non-negativity and symmetry of $\operatorname{\delta}$. To show the identity of indiscernibles, we assume that $\operatorname{\delta}^{*}([x],[y])=0$. From the identity $\delta^{*}([x],[y])=\delta(x^{*},y^{*})$ follows $\operatorname{\delta}(x^{*},y^{*})=0$. This implies that $x^{*}$ and $y^{*}$ are warping identical, that is $x^{*}\sim y^{*}$. By Prop. 2 we have the following relations $[x^{*}]=[y^{*}]$, $[x]=[x^{*}]$, and $[y]=[y^{*}]$. Combining these relations gives $[x]=[y]$. This shows that $\operatorname{\delta}^{*}$ satisfies the identity of indiscernibles. Hence, $\delta^{*}$ is a semi-metric. ∎

A.5 Warping-Invariance

This section shows that the canonical extension of the twi-distance is warping-invariant. A distance function $d:{\mathcal{T}}\times{\mathcal{T}}\rightarrow{\mathbb{R}}_{\geq 0}$ is warping-invariant if

[TABLE]

for all time series $x,y,x^{\prime},y^{\prime}\in{\mathcal{T}}$ with $x\prec x^{\prime}$ and $y\prec y^{\prime}$. The twi-distance $\delta^{*}$ extends to a distance on ${\mathcal{T}}$ by virtue of

[TABLE]

We call $\operatorname{\delta}^{\sim}$ the canonical extension of $\operatorname{\delta}^{*}$.

Section 2.4, Theorem 6.

The canonical extension $\operatorname{\delta}^{\sim}$ of the twi-distance $\operatorname{\delta}^{*}$ is warping-invariant.

Proof.

Let $x,x^{\prime},y,y^{\prime}\in{\mathcal{T}}$ be time series such that there are common compressions $u\prec x,x^{\prime}$ and $v\prec y,y^{\prime}$. We show that $\delta^{\sim}(x,y)=\delta^{\sim}(x^{\prime},y^{\prime})$. Suppose that $x^{*}$ and $y^{*}$ are the condensed forms of $u$ and $v$, respectively. By repeatedly applying Prop. 12 we obtain $x,x^{\prime}\succ x^{*}$ and $y,y^{\prime}\succ y^{*}$. From Prop. 3 follows that $[x]=[x^{\prime}]$ and $[y]=[y^{\prime}]$. This gives $\delta^{\sim}(x,y)=\delta^{*}([x],[y])=\delta^{*}([x^{\prime}],[y^{\prime}])=\delta^{\sim}(x^{\prime},y^{\prime})$. The proof is complete. ∎