Time-warping invariants of multidimensional time series

TL;DR

This paper explores time-warping invariants of multidimensional time series, linking them to iterated sums called quasisymmetric functions, and develops an algebraic framework for these features.

Contribution

It introduces a novel algebraic approach to characterize time-warping invariants using quasisymmetric functions, providing foundational properties for feature extraction.

Findings

Identifies quasisymmetric functions as invariants under time-warping.

Provides an algebraic framework for these invariants.

Lays groundwork for invariant feature extraction in time series analysis.

Abstract

In data science, one is often confronted with a time series representing measurements of some quantity of interest. Usually, as a first step, features of the time series need to be extracted. These are numerical quantities that aim to succinctly describe the data and to dampen the influence of noise. In some applications, these features are also required to satisfy some invariance properties. In this paper, we concentrate on time-warping invariants. We show that these correspond to a certain family of iterated sums of the increments of the time series, known as quasisymmetric functions in the mathematics literature. We present these invariant features in an algebraic framework, and we develop some of their basic properties.

Figures

Equations

$$x=(x_0,x_1,\dots ,x_N)\in ({\mathbb{R}}^d{)}^N,$$

$$y_n^{(\ell )}=x_{h^{(\ell )}(n)}+w_n^{(\ell )},n=1,\dots ,M,\ell =1,\dots ,L.$$

$$\begin{array}{r}\sum _i(x_i-x_{i-1}{)}^2,\sum _{i_1<i_2}(x_{i_1}-x_{i_1-1})(x_{i_2}-x_{i_2-1}),\sum _{i_1\le i_2}(x_{i_1}-x_{i_1-1})(x_{i_2}-x_{i_2-1}),\end{array}$$

$$\sum _i{Y}_i,\sum _i(Y_i{)}^2,\sum _{i_1<i_2}{Y}_{i_1}{Y}_{i_2},\sum _{i_1\le i_2}{Y}_{i_1}{Y}_{i_2},$$

$$[1^{c_1}]\cdots [1^{c_k}]⟷M_{(c_1,\dots ,c_k)}:=\sum _{i_1<\cdots <i_k}(Y_{i_1}{)}^{c_1}\cdots (Y_{i_k}{)}^{c_k}.$$

$$[1]\ast [1^3][1^7]$$

$$\left(\sum _i{Y}_i\right)\cdot \left(\sum _{i_1<i_2}(Y_{i_1}{)}^3(Y_{i_2}{)}^7\right)=\sum _{i_1<i_2<i_3}{Y}_{i_1}(Y_{i_2}{)}^3(Y_{i_3}{)}^7+\sum _{i_1<i_2<i_3}(Y_{i_1}{)}^3{Y}_{i_2}(Y_{i_3}{)}^7$$

$$⟨[1^{c_1}]\cdots [1^{c_k}],\mathrm{ISS}(x)⟩:=\sum _{i_1<\cdots <i_k}(\Delta x_{i_1}{)}^{c_1}\cdots (\Delta x_{i_k}{)}^{c_k}.$$

$$⟨[1^{p_1}]\cdots [1^{p_k}],\mathrm{ISS}(x)⟩\cdot ⟨[1^{q_1}]\cdots [1^{q_l}],\mathrm{ISS}(x)⟩$$

$$ua\star vb:=(u\star vb)a+(ua\star v)b+(u\star v)[ab],$$

$$3\star 4[12]=34[12]+43[12]+4[12]3+[34][12]+4[123].$$

$$\delta (w):=w\otimes e+e\otimes w+\sum _{i=1}^{n-1}{w}_1\cdots w_i\otimes w_{i+1}\cdots w_n,$$

$$\alpha (w_1\cdots w_n)=(-1{)}^n\sum _{I\in \mathcal{C}(n)}I[w_n\cdots w_1].$$

$$I[w]:=[w_1\cdots w_{i_1}][w_{i_1+1}\cdots w_{i_1+i_2}]\cdots [w_{i_1+\cdots +i_{p-1}+1}\cdots w_n].$$

$$v\mathtt{i}\text{shuffle}w\mathtt{j}:=(v\text{shuffle}w\mathtt{j})\mathtt{i}+(v\mathtt{i}\text{shuffle}w)\mathtt{j},$$

$$G(t):=\sum _{n\ge 0}{t}^n\dim T(\mathfrak{A}{)}_n$$

$$\dim T(\mathfrak{A}{)}_n=\sum _{(i_1,\dots ,i_p)\in \mathcal{C}(n)}(\genfrac{}{}{0ex}{}{d-1+i_1}{i_1})\cdots (\genfrac{}{}{0ex}{}{d-1+i_p}{i_p}).$$

$$\left(\genfrac{}{}{0ex}{}{d-1+i}{i}\right)=\frac{d(d+1)\cdots (d+i-1)}{i!}=\frac{1}{i!}(d{)}_i,$$

$${}_1{F}_0(d;t)=1+\sum _{i=1}^{\infty }(d{)}_i\frac{t^i}{i!}=(1-t{)}^{-d}.$$

$$G(t)=\sum _{n=0}^{\infty }{t}^n\sum _{(i_1,\dots ,i_p)\in \mathcal{C}(n)}\frac{(d{)}_{i_1}\cdots (d{)}_{i_p}}{i_1!\cdots i_p!}$$

$$=\sum _{p=0}^{\infty }{\left((1-t{)}^{-d}-1\right)}^p=\frac{(1-t{)}^d}{2(1-t{)}^d-1}.$$

$$c\cdot c^{\prime }=\sum _{w\in T(\mathfrak{A})}\sum _{uv=w}⟨u,c⟩⟨v,c^{\prime }⟩w=\sum _{w\in T(\mathfrak{A})}⟨\delta (w),c\otimes c^{\prime }⟩w.$$

$$⟨u\star v,c⟩=⟨u\otimes v,{\Delta }_{\mathrm{qsh}}(c)⟩=⟨u\otimes v,c\otimes c⟩,$$

$${\Delta }_{\mathrm{qsh}}(w):=\sum _{u,v\in T(\mathfrak{A})}⟨u\star v,w⟩u\otimes v.$$

$$\Sigma \left([1^{k_1}][1^{k_2}]\cdots [1^{k_n}]\right):=\sum _{1\le i_1<\cdots <i_n}(Y_{i_1}{)}^{k_1}\cdots (Y_{i_n}{)}^{k_n}=:M_{(k_1,\dots ,k_n)}.$$

$$\Sigma \left([1]\right)\cdot \Sigma \left([1^3][1^7]\right)=\left(\sum _i{Y}_i\right)\cdot \left(\sum _{i_1<i_2}(Y_{i_1}{)}^3(Y_{i_2}{)}^7\right)$$

$$=\sum _{i_1<i_2<i_3}{Y}_{i_1}(Y_{i_2}{)}^3(Y_{i_3}{)}^7+\sum _{i_1<i_2<i_3}(Y_{i_1}{)}^3{Y}_{i_2}(Y_{i_3}{)}^7+\sum _{i_1<i_2<i_3}(Y_{i_1}{)}^3(Y_{i_2}{)}^7{Y}_{i_3}$$

$$+\sum _{i_1<i_2}(Y_{i_1}{)}^4(Y_{i_2}{)}^7+\sum _{i_1<i_2}(Y_{i_1}{)}^3(Y_{i_2}{)}^8$$

$$=\Sigma \left([1][1^3][1^7]+[1^3][1][1^7]+[1^3][1^7][1]+[1^4][1^7]+[1^3][1^8]\right)$$

$$=\Sigma \left([1]\star [1^3][1^7]\right),$$

$$ua\dot{\succ }vb:=(ua\star v)b,ua\dot{\prec }vb:=(u\star vb)a,ua\diamond vb:=(u\star v)[ab],$$

$$\begin{array}{rl}3\dot{\succ }4[12]& =34[12]+43[12]+[34][12]\\ 3\dot{\prec }4[12]& =4[12]3\\ 3\diamond 4[12]& =4[123].\end{array}$$

$$u\mathtt{a}\succ v\mathtt{b}:=(u\mathtt{a}\text{shuffle}v)\mathtt{b},u\mathtt{a}\prec v\mathtt{b}:=(u\text{shuffle}v\mathtt{b})\mathtt{a},$$

Full text

In some applications, these features are also required to satisfy some invariance properties. In this paper, we concentrate on time-warping invariants. We show that these correspond to a certain family of iterated sums of the increments of the time series, known as quasisymmetric functions in the mathematics literature. We present these invariant features in an algebraic framework, and we develop some of their basic properties.

1. Motivation

Given a discrete time series

[TABLE]

where $N\geq 1$ is some arbitrary time horizon, our foremost, and original, motivation stems from the desire to extract features from $x$ that are invariant to time warping.

The precise definition of the latter will be given in Section 4, but Figure 1 illustrates what we mean by time warping: the time series is allowed to “stand still” or to “stutter” (this term is used in [47]), which means that $x$ has repetitions of values at consecutive time steps (here at time $t=3$).

Remark 1.1**.**

In this section we consider the notationally simpler case $d=1$, that is, when $x\in\R^{N}$.

Our interest is prompted, on the one hand by the extensive literature on the dynamic time warping (DTW) distance [5], a distance on discrete time series that is invariant to time warping. In [47] it is stated that

“the time warping distance …does not lead to any natural features”.

Our work aims to provide those missing “natural” features.

On the other hand the following example illustrates where such invariant features will become useful.

Example 1.2**.**

Assume that there is a deterministic time series $x\in\R^{N}$ which models some “prototype” evolution of a quantity, say the prototype heartbeat in a patient’s ECG. This prototype is unknown, but one records a lot of samples of it run at different speeds and contaminated by noise (compare [6]). A model for these observations is then

[TABLE]

Here $L$ is the number of observations, $M\geq N$ is the time horizon we allow the prototype to be “spread out” over, $h^{(\ell)}:[1,\ldots,M]\to[1,\ldots,N]$ are unknown non-decreasing, surjective time changes and $w^{(\ell)}_{n}$ are independent and identically distributed (iid) random walks. The goal is to recover $x$ (up to time warping).

The currently used method [6, 31, 33], consists in first trying to align the different samples, i.e., to estimate the time-changes $h^{(\ell)}$, and to average afterwards. This seems to work well in regimes where the noise $w^{(\ell)}$ is small (large signal-to-noise ratio), but will break down if this is not the case.

Guided by invariant methods in cryo-EM [4] we then propose the following procedure.

1. (1)

Calculate features of $y^{(\ell)}$ that do not see time warpings. 2. (2)

Average those features over the independent samples, giving the law of large numbers a chance to cancel out the noise and getting an approximation of the features of $x$. 3. (3)

Invert the averaged features to arrive at a candidate for $x$.

Our approach to Step (1) is new and will be presented in this paper. Step (2) and Step (3) will be addressed in future work.

A moment’s thought reveals that iterated-sums of the increments of $x$ are invariant in the desired sense. For example, the simple sum $\sum_{i}(x_{i}-x_{i-1})$ or the more complex expressions

[TABLE]

are features of the time series that do not change when warping time, i.e., when repetitions of points, $x_{i}=x_{i+1}=\cdots=x_{i+j}$ occur in $x$.

Remark 1.3**.**

To accommodate repetition of points, here we have conveniently written the sum over an unspecified set of time-points. We can think of the sum taken over ${\N_{+}}$, with $x$ being extended constantly as $x_{N}$ after time $N$.

However, two questions immediately emerge

1. (A)

The three expressions in (1) are already linearly dependent (adding the first and second sum gives the third). How to store only linearly independent expressions? 2. (B)

Do iterated-sums of increments give all (polynomial) time warping invariants?

Regarding the first item, it turns out that the above iterated-sums expressions are reminiscent of quasisymmetric functions [39]. Consider the space $\R\langle Y_{1},Y_{2},Y_{3},\dots\rangle$ of formal power series in ordered commuting variables $Y_{1},Y_{2},Y_{3},\dots$. By definition, a power series (of finite degree) $Q\in\R\langle Y_{1},Y_{2},Y_{3},\dots\rangle$ is a quasisymmetric function if for all $n\geq 1$, all $i_{1}<\cdots<i_{n}$, all $j_{1}<\cdots<j_{n}$ and all $\alpha_{1},\dots,\alpha_{n}\geq 1$, the coefficient of the monomial $(Y_{i_{1}})^{\alpha_{1}}\cdots(Y_{i_{n}})^{\alpha_{n}}$ in $Q$ is equal to the one of $(Y_{j_{1}})^{\alpha_{1}}\cdots(Y_{j_{n}})^{\alpha_{n}}$. First examples are

[TABLE]

and we see that the invariants given above follow from the evaluation of these quasisymmetric functions at $Y_{1}\mapsto x_{1}-x_{0},Y_{2}\mapsto x_{2}-x_{1},\dots,Y_{N}\mapsto x_{N}-x_{N-1}$, and $Y_{i}\mapsto 0$ for $i\geq N+1$.

Different linear basis for quasisymmetric functions are known. The one of monomial quasisymmetric functions of [39] is indexed by compositions of integers. Anticipating the multidimensional case, we write a composition $c_{1}+\cdots+c_{k}=n$ as $[\mathtt{1}^{c_{1}}]\cdots[\mathtt{1}^{c_{k}}]$, and obtain the correspondence

[TABLE]

Quasisymmetric functions are a refinement of symmetric functions and form a commutative unital algebra. The product is just the polynomial product in the power series representation. It amounts to a so-called quasi-shuffle product (see Section 2) in the representation as compositions. For example, the abstract quasi-shuffle product

[TABLE]

corresponds to the concrete product of power series

[TABLE]

The latter equality follows by case distinction for sums over the three indexing variables, which amounts to a summation-by-parts formula. The last two terms in the above product reflect the fact that multiplying sums requires the inclusion of sums over diagonal terms.

It is natural to store the iterated-sums invariants of the discrete time series $x$ as a linear map $\operatorname{ISS}(x)$ on the quasi-shuffle algebra of compositions, by defining the pairing

[TABLE]

Here $\Delta x_{i}:=x_{i}-x_{i-1}$ for $1\leq i\leq N$, and as above we extend $x$ constantly, so that $\Delta x_{i}:=0$ for $i\geq N+1$. From the correspondence between the product of power series and the quasi-shuffle product of compositions mentioned above we deduce that

[TABLE]

Hence, $\operatorname{ISS}(x)$, which we call iterated-sums signature, is an algebra morphism (from the quasi-shuffle algebra to the underlying base field $\mathbb{F}$). Since compositions form a linear basis, this answers Question (A) above – in the case $d=1$. We will come back to Question (B) in Section 4.

The commutative algebra of quasisymmetric functions is the free quasi-shuffle algebra over one generator and it is - as we just saw - the correct framework to store iterated-sums for a one-dimensional time-series. The appropriate generalisation of this algebra to arbitrary dimension $d\geq 1$, that is, the free quasi-shuffle algebra over $d$ generators, was carried out by Hoffman [27].

The aforementioned amounts to saying that iterated-sums signature $\operatorname{ISS}(x)$ is an element of the dual space of the quasi-shuffle algebra over $d$ generators. It can therefore be represented as an infinite word series with iterated-sums of the time series $x$ as coefficients. Its compatibility with the quasi-shuffle product together with the fact that the latter can be seen as a deformation of the classical shuffle product [18] suggests to consider $\operatorname{ISS}(x)$ as a discrete analog of Chen’s iterated-integrals signature over continuous curves [10, 44]. The latter plays an important role in the theory of controlled ordinary differential equations (ODEs), stochastic analysis and Lyons’ theory of rough paths [19, 37]. Such a large spectrum of applications reflects the important property of iterated-integrals to provide - in some sense - a complete representation of a curve, so that arbitrary functionals on curves should be well approximated by functions on its signature. There is a caveat though. Iterated integrals are tailor made to approximate functionals that stem from controlled ODEs. But as is quickly realised, this does not mean that the iterated-integrals signature is an optimal representation for other input-output systems. For example, since a controlled ODE - and hence also the signature - cannot see tree-like excursions, the iterated-integrals signature of a one-dimensional path reveals nothing about the path, except for its increment.There are several procedures to circumvent this shortcoming, and to obtain information even about tree-like parts of a curve using signature. These procedures usually consists of lifting the path to a higher-dimensional curve and calculating the signature of it. The aforementioned limitations of the iterated-integrals signature with respect to tree-like paths prompts us to propose instead the use of “discrete time signature” $\operatorname{ISS}(x)$, which, instead of storing iterated-integrals, gathers iterated-sums.

Remark 1.4**.**

For the precise definition of “tree-like” see [24]; but one can think of a curve that completely “tracks back”. In particular in dimension $1$, every curve that has coinciding start- and endpoint is tree-like.

The paper is organised as follows. Section 2 recalls the notion of quasi-shuffle Hopf algebra and quasisymmetric functions. In Section 3 we introduce the iterated-sums signature and show its character property with respect to the quasi-shuffle Hopf algebra. Moreover, we show that Chen’s property is satisfied, but that Chow’s Theorem does not hold. Hence, while mirroring the setup of Chen’s iterated-integrals signature to some extent, interesting differences emerge. It turns out that our description of the iterated-sums signature is nicely related to the work [42] on “multidimensional” generalisation of quasisymmetric functions, and we dwell on this briefly in Remark 3.5. In Section 4 we show the iterated-sums signature contains (almost) all time warping invariants. In Section 5 we use a specific Hopf algebra isomorphism, known as Hoffman’s exponential, to relate the iterated-sums signature to Chen’s iterated-integrals signature (of an infinite-dimensional path). This includes in particular relating the continuous and discrete area operations.

In the following all algebraic structures are defined over a base field $\mathbb{F}$ of characteristic zero. The reader is invited to think of the field $\mathbb{F}$ as the reals, $\mathbb{F}=\R$, or the complex numbers, $\mathbb{F}=\mathbb{C}$, throughout.

We denote ${\N}\coloneqq\{0,1,2,\dots\}$ and ${\N_{+}}\coloneqq\{1,2,\dots\}$. All (co)algebras are (co)unital and (co)associative unless otherwise stated. For details on Hopf algebras the reader is referred to [9, 26, 36, 40, 46].

2. Quasi-shuffle Hopf algebra

The notion of quasi-shuffle product appeared first in a 1972 article by Cartier [8]. Its Hopf algebraic relevance was explored in the 1979 paper [41]. Two decades later, Hoffman [27] provided a comprehensive account of the quasi-shuffle product in a Hopf algebraic framework. Meanwhile, quasi-shuffle products appeared under different names, i.e., modified shuffle product [20, 34], sticky-shuffle [29, 30], overlapping shuffle [25], stuffle and harmonic product [48].

We recall the inductive definition of the quasi-shuffle product following Hoffman [27]. See also [7, 16]. Our starting point is the alphabet $A=\{\mathtt{1},\mathtt{2},\dotsc,\mathtt{d}\}$, which we augment to a free commutative semigroup, $\mathfrak{A}$, by defining a commutative product denoted by square brackets, $[--]\colon\mathfrak{A}\times\mathfrak{A}\to\mathfrak{A}$. For example, the product between the letters $\mathtt{1},\mathtt{2}\in A$ is written $[\mathtt{1}\mathtt{2}]=[\mathtt{2}\mathtt{1}]$. Any iteration of the product in $\mathfrak{A}$ can be simplyfied to an expression containing a single pair of brackets, that is, $[\mathtt{i_{1}}\cdots\mathtt{i_{n}}]\coloneqq[\mathtt{i_{1}}[\cdots[\mathtt{i_{n-1}}\mathtt{i_{n}}]]\cdots]$. For instance, $[\mathtt{1}\mathtt{2}\mathtt{3}]=[\mathtt{1}[\mathtt{2}\mathtt{3}]]$ in $\mathfrak{A}$. Elements in the tensor algebra $T(\mathfrak{A})$ over (the vector space spanned by) $\mathfrak{A}$ are denoted by words, i.e., we denote the tensor product by concatenation, or juxtaposition of basis elements. The neutral element for this product is the empty word, denoted by $e$. The augmentation ideal is defined by $T_{+}(\mathfrak{A})\coloneqq\oplus_{n>0}{\mathfrak{A}}^{\otimes n}$ such that $T(\mathfrak{A})=\mathbb{F}e\oplus T_{+}(\mathfrak{A})$.

The commutative quasi-shuffle product $m_{\star}\colon T(\mathfrak{A})\otimes T(\mathfrak{A})\to T(\mathfrak{A})$, $u\star v:=m_{\star}(u,v)$, is introduced by inductively defining $e\star u\coloneqq u\eqqcolon u\star e$, for all $u\in T(\mathfrak{A})$, and

[TABLE]

for $u,v\in T(\mathfrak{A})$ and $a,b\in\mathfrak{A}$. For example, $\mathtt{2}\star\mathtt{3}=\mathtt{2}\mathtt{3}+\mathtt{3}\mathtt{2}+[\mathtt{2}\mathtt{3}]$ and

[TABLE]

The tensor algebra is naturally graded by the length of words, $\ell(w_{1}\dotsm w_{n})=n$ for $w_{1}\dotsm w_{n}\in T(\mathfrak{A})$. However, in light of the new product (2), which is not homogenous with respect to the number of letters, we introduce the weight grading on $T(\mathfrak{A})$, denoted $|\cdot|$, by declaring that $|e|=0$, $|\mathtt{a}|=1$ for all $\mathtt{a}\in A$ and $|[pq]|=|p|+|q|$ for all $p,q\in\mathfrak{A}$. Finally, for a word $w=w_{1}\dotsm w_{n}\in T(\mathfrak{A})$ we define its weight to be $|w|=|w_{1}|+\dotsm+|w_{n}|$.

Let $\delta\colon T(\mathfrak{A})\to T(\mathfrak{A})\otimes T(\mathfrak{A})$ denote the deconcatenation coproduct defined on a nonempty word $w=w_{1}\cdots w_{n}\in T(\mathfrak{A})$ by

[TABLE]

and $\delta(e)=e\otimes e$. It turns $T(\mathfrak{A})$ into a connected graded coalgebra, for both the length and weight grading. For any word $w\in T_{+}(\mathfrak{A})$ the reduced coproduct is defined by $\delta^{\prime}(w)\coloneqq\delta(w)-w\otimes e-e\otimes w$. “Sweedler’s notation” will be employed for both coproducts: $\delta(w)\eqqcolon\sum_{(w)}w_{(1)}\otimes w_{(2)}$ and $\delta^{\prime}(w)\eqqcolon\sideset{}{{}^{\prime}}{\sum}_{(w)}w_{(1)}\otimes w_{(2)}$. The canonical counit map $\varepsilon\colon T(\mathfrak{A})\to\mathbb{F}$ is defined to be $\varepsilon(\lambda e)=\lambda\in\mathbb{F}$ and zero on $T_{+}(\mathfrak{A})$. In [27] Hoffman showed the following

Theorem 2.1** (Quasi-shuffle Hopf algebra).**

1. $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star,\delta,\varepsilon,|\cdot|)$ is a graded, connected, commutative, non-cocommutative Hopf algebra.

2. The antipode $\alpha\colon H_{\mathrm{qsh}}\to H_{\mathrm{qsh}}$ is given by

[TABLE]

Here $\mathcal{C}(n)$ is the set of all compositions of the integer $n$, i.e., tuples $(i_{1},\dotsc,i_{p})$ of positive integers such that $i_{1}+\dotsb+i_{p}=n$. Given $I=(i_{1},\dotsc,i_{p})\in\mathcal{C}(n)$ and a word $w=w_{1}\dotsm w_{n}\in T(\mathfrak{A})$ of length $\ell(w)=n>0$, we define a new word $I[w]\in T(\mathfrak{A})$ by

[TABLE]

Here (as well as later) we are using the suitable convention that $[a]:=a$ for all $a\in\mathfrak{A}$.

Remark 2.2** (Shuffle Hopf algebra).**

If the semigroup $\mathfrak{A}$ is trivial, i.e., if $[\mathtt{i}\mathtt{j}]=0$ for any letters $\mathtt{i},\mathtt{j}\in A$, then the quasi-shuffle product (2) reduces to Chen’s commutative shuffle product on $T(A)$:

[TABLE]

for $v,w\in T(A)$ and $\mathtt{i},\mathtt{j}\in A$. Observe that in this case $|w|=\ell(w)$ for any word and $H_{\shuffle}=(T(\mathfrak{A}),\shuffle,\delta,\varepsilon,\ell)$ is the classical shuffle Hopf algebra over the alphabet $A$. From (5) it follows that the antipode on $H_{\shuffle}$ is given by $\alpha(\mathtt{i}_{1}\dotsm\mathtt{i}_{n})=(-1)^{n}\mathtt{i}_{n}\dotsm\mathtt{i}_{1}$. See [46] for a comprehensive account on $H_{\shuffle}$.

Remark 2.3** (A remark on dimensions.).**

There is a simple way of computing the Hilbert series

[TABLE]

of $T(\mathfrak{A})$, where $T(\mathfrak{A})_{n}\coloneqq\mathbb{F}\{w:|w|=n\}$ is the homogeneous (for the weight grading) component of degree $n$ of the quasi-shuffle algebra. It is not hard to see that all such words are of the form $I[\mathtt{a}_{1}\dotsm\mathtt{a}_{n}]$ for some composition $I=(i_{1},\dotsc,i_{p})\in\mathcal{C}(n)$ and letters $\mathtt{a}_{1},\dotsc,\mathtt{a}_{n}\in A$, in the notation of Theorem 2.1. In total, in each block of size $i_{1},i_{2},\dotsc,i_{p}$ we are allowed to put a symmetric monomial of length $i_{j}$ of which there are exactly $\binom{d-1+i_{j}}{i_{j}}$ – this is the dimension of the degree-$i_{j}$ part of the symmetric algebra $S(A)$. Therefore

[TABLE]

A simple computation shows that in fact

[TABLE]

where the Pochhammer symbol (or rising factorial) appears on the righthand side. It is well known that their exponential generating function equals the hypergeometric function

[TABLE]

Therefore

[TABLE]

The coeffficients of these Hilbert series can be found in column $d$ of the OEIS sequence A261780.

Define the scalar product $\langle-,-\rangle\colon T(\mathfrak{A})\otimes T(\mathfrak{A})\to\mathbb{F}$ for any words $u,v\in T(\mathfrak{A})$ by $\langle u,v\rangle\coloneqq 1$ if $u=v$ and zero else. It permits to identify the graded dual of $H_{\mathrm{qsh}}$ as word series, i.e., $c=\sum_{w\in T(\mathfrak{A})}\langle w,c\rangle w\in T(\mskip-4.0mu(\mathfrak{A})\mskip-4.0mu)\eqqcolon H_{\mathrm{qsh}}^{*}$, which is a non-commutative (topological) Hopf algebra with concatenation as convolution product, denoted by $m_{\centerdot}\colon H_{\mathrm{qsh}}^{*}\otimes H_{\mathrm{qsh}}^{*}\to H_{\mathrm{qsh}}^{*}$, $c\centerdot c^{\prime}\coloneqq m_{\centerdot}(c\otimes c^{\prime})$, and de-quasi-shuffling as coproduct [27]. In more concrete terms, this means that given two such series $c,c^{\prime}\in T(\mskip-4.0mu(\mathfrak{A})\mskip-4.0mu)$ their convolution product $c\centerdot c^{\prime}\coloneqq m_{\mathbb{F}}(c\otimes c^{\prime})\delta:H_{\mathrm{qsh}}\to\mathbb{F}$ may be written as

[TABLE]

Of particular interest are characters, i.e., algebra morphisms $c\in H_{\mathrm{qsh}}^{*}$. They satisfy $\langle e,c\rangle=1$ and $\langle u\star v,c\rangle=\langle u,c\rangle\langle v,c\rangle$, for $u,v\in H_{\mathrm{qsh}}$. The first property requires that the coefficient $\langle e,c\rangle=1$ and the second is equivalent to $c$ being group-like in $H_{\mathrm{qsh}}^{*}$, which means that for $u,v\in H_{\mathrm{qsh}}$

[TABLE]

where the de-quasi-shuffling coproduct is defined on words by

[TABLE]

The set of characters, denoted by $\mathscr{G}$, forms a group with the inverse $c^{-1}=c\circ\alpha$. The corresponding Lie algebra, $\mathfrak{g}\subset H_{\mathrm{qsh}}^{*}$, consists of so-called infinitesimal characters, which map the empty word and any non-trivial product in $H_{\mathrm{qsh}}$ to zero. One can define the exponential map as a power series with respect to the convolution product which maps $\mathfrak{g}$ bijectively to $\mathscr{G}$, i.e., $\exp^{\centerdot}(f):=\varepsilon+\sum_{j>0}\frac{1}{j!}f^{\centerdot j}\in\mathscr{G}$. Because $T(\mathfrak{A})$ is a graded connected Hopf algebra, this expression becomes a finite sum when evaluated on homogeneous elements of $T(\mathfrak{A})$, so we do not have to deal with convergence issues. Its inverse is the logarithm, $\log^{\centerdot}(\varepsilon+(c-\varepsilon))=\sum_{i\geq 1}\tfrac{(-1)^{i-1}}{i}(c-\varepsilon)^{\centerdot i}\in\mathfrak{g}$. Again, the sum applied to any word $w\in T(\mathfrak{A})$ terminates after $|w|$ terms, as $(c-\varepsilon)(e)=0$.

Notation 2.4**.**

We introduce a particular notation for words in $T(\mathfrak{A})$, which will be useful in the sequel. The convention to identify $[a]\coloneqq a$, for $a\in\mathfrak{A}$, permits to write any word in $T(\mathfrak{A})$ as a concatenation of brackets, i.e., $w=[u_{1}]\cdots[u_{k}]\in T(\mathfrak{A})$, for $u_{1},\ldots,u_{k}\in\mathfrak{A}$.

We come back to the setting of the introductory section with only a single letter, $A=\{\mathtt{1}\}$. Then, in each degree $n$, $T(\mathfrak{A})$ has a single word of length one, $[1^{n}]\in\mathfrak{A}$, and any basis element (or word) is of the form $w=[\mathtt{1}^{k_{1}}][\mathtt{1}^{k_{2}}]\dotsm[\mathtt{1}^{k_{n}}]$ for some integers $k_{1},\dotsc,k_{n}>0$. It is easy to see that then the tuple $(k_{1},\dotsc,k_{n})$ is a composition of the integer $|w|$ of length $n=\ell(w)$. In [27] Hoffman describes a unital algebra isomorphism $\Sigma$ between the quasi-shuffle algebra $H_{\mathrm{qsh}}$, for $A=\{\mathtt{1}\}$, and the algebra $\mathrm{QSym}$ of quasisymmetric functions in the ordered set of commuting variables $\{Y_{i}\}_{i\in\mathbb{N}_{+}}$ [21], defined by taking a word in $T(\mathfrak{A})$ to an iterated sum

[TABLE]

Here $\Sigma(e)=M_{0}=1$. Then, the correspondence of the introduction is explicitly given by

[TABLE]

where the second equality is an example of summation-by-parts for products of iterated sums.

The $M_{(k_{1},\ldots,k_{n})}$ of (6) are the monomial quasisymmetric functions, which form a basis for $\mathrm{QSym}$. The Hopf algebra $\mathrm{QSym}$ is a generalisation of the classical Hopf algebra $\mathrm{Sym}$ of symmetric functions. It was defined and studied by Gessel [21], based on earlier work by Stanley, and plays a rather distinguished role in modern algebraic combinatorics, with ramifications into several other fields of mathematics. Its graded dual is known as the connected graded cocommutative Hopf algebra $\mathrm{NSym}$ of noncommutative symmetric functions. The iterated-sums signature corresponding to a one dimensional discrete time series, alluded to in the first section, is an element in $\mathrm{NSym}$. Further below, in Section 3, we consider the multidimensional generalisation of quasisymmetric functions (of level $d$ in the terminology of [42]) and its corresponding iterated-sums signature. We close this section by mentioning that Malvenuto’s and Reutenauer’s Hopf algebra of permutations [39] plays an important part in the understanding of the relation between the objects $\mathrm{Sym}$, $\mathrm{QSym}$ and $\mathrm{NSym}$. The interested reader is referred to [2, 1] and to [36] for a readable introduction, including a brief historical overview.

2.1. Half-shuffles

Aiming at understanding the discrete analog of the ${\mathrm{area}}$ operation (to be introduced further below), we take a more refined approach at the quasi-shuffle product by observing that $m_{\star}$ may be split into three products, i.e., left and right half-shuffles and a third product

[TABLE]

so that $u\star v=u\mathbin{\dot{\prec}}v+u\mathbin{\dot{\succ}}v+u\diamond v$. For instance (c.f. Example 3)

[TABLE]

Noticing the particular relation $ua\mathbin{\dot{\succ}}vb=vb\mathbin{\dot{\prec}}ua$ which is equivalent to $m_{\star}$ being commutative, it is not hard to show that the quasi-shuffle algebra $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star)$ becomes a commutative tridendriform algebra, $(T(\mathfrak{A}),{\mathbin{\dot{\prec}}},{\mathbin{\dot{\succ}}},{\diamond})$, as defined by Loday and Ronco [35].

Remark 2.5**.**

A similar splitting holds for the shuffle algebra in Remark 2.2. We can write the shuffle product $m_{\shuffle}$ on $T(A)$ as a sum of the two half-shuffles

[TABLE]

so that $u\mathtt{a}\shuffle v\mathtt{b}=u\mathtt{a}\prec v\mathtt{b}+u\mathtt{a}\succ v\mathtt{b}$. Again, we check quickly that the commutativity of the shuffle product is equivalent to $u\mathtt{a}\succ v\mathtt{b}=v\mathtt{b}\prec u\mathtt{a}$. In fact, the triple $(T(A),{\prec},{\succ})$ is also known as a commutative dendriform or Zinbiel algebra.

2.2. Hoffman’s exponential

Shuffle and quasi-shuffle Hopf algebras are more tightly related than Remark 2.2 may adumbrate. Indeed, Hoffman proved in [27] that $H_{\shuffle}=(T(\mathfrak{A}),\shuffle,\delta)$ and $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star,\delta)$ are isomorphic as Hopf algebras. We briefly recall this result. Let $T(\mathfrak{A})$ be equipped with the commutative shuffle product $m_{\shuffle}\colon T(\mathfrak{A})\otimes T(\mathfrak{A})\to T(\mathfrak{A})$ inductively defined by $u[a]\shuffle v[b]\coloneqq(u\shuffle v[b])[a]+(u[a]\shuffle v)[b]$, for $u,v\in T(\mathfrak{A})$ and $a,b\in\mathfrak{A}$. The empty word, $e$, is the unit for this product. Recall the notation $I[w]$ introduced in Theorem 2.1.

Theorem 2.6** (Hoffman’s isomorphism).**

[27]** There exists a Hopf algebra isomorphism $\Phi_{\mathrm{H}}\colon(T(\mathfrak{A}),\shuffle,\delta)\to(T(\mathfrak{A}),\star,\delta)$, given explicitly by the so-called Hoffman exponential

[TABLE]

Its inverse also admits an explicit expression, namely the Hoffman logarithm

[TABLE]

Some examples: $\Phi_{\mathrm{H}}([\mathtt{i}])=[\mathtt{i}]$ and for the words $[\mathtt{1}][\mathtt{2}]\in T(\mathfrak{A})$ and $[\mathtt{1}][\mathtt{23}][\mathtt{4}]\in T(\mathfrak{A})$ we find

[TABLE]

In the second example, the terms correspond to the compositions $(1,1,1)$, $(2,1)$, $(1,2)$ and $(3)$ of the integer $3$, in that order. Recall that the particular Notation 2.4 for words $w=[u_{1}]\cdots[u_{k}]\in T(\mathfrak{A})$, for $u_{1},\ldots,u_{k}\in\mathfrak{A}$, is in place. Also, note that the number of letters in each of the terms corresponds to the length of the composition. The reader is referred to [27, 28] for more details. See also [16] for an application in stochastic analysis.

In Section 5 we will show that $\Phi_{\mathrm{H}}$ is nicely compatible with comparing the iterated-sums signature on one side with the iterated-integrals signature on the other. The following two lemmas are going to be used in Section 5.1, where we address the area operation in the context of the iterated-sums signature.

Lemma 2.7**.**

The image of any nonempty word $w=w_{1}\dotsm w_{n}\in T(\mathfrak{A})$ under Hoffman’s isomorphism can be split into two parts as follows:

[TABLE]

where the remainder term

[TABLE]

The verification of the lemma is left to the reader. This splitting of Hoffman’s isomorphism implies the following important result.

Lemma 2.8**.**

Let $u\in T(\mathfrak{A})$ and $a,b\in\mathfrak{A}$. Then

[TABLE]

Proof 2.9**.**

From Lemma 2.7 and linearity of $\Phi_{\mathrm{H}}$, we deduce that

[TABLE]

Since the semigroup $\mathfrak{A}$ is commutative, for any composition $I=(i_{1},\dotsc,i_{p})\in\mathcal{C}(n)$ with $i_{p}\geq 2$ we have that

[TABLE]

Therefore, the equality $R_{\mathrm{H}}(u[a][b])=R_{\mathrm{H}}(u[b][a])$ holds, which implies the identity (12).

3. Iterated-sums signatures

We consider a discrete time series $x\in(\mathbb{F}^{d})^{N}$ as an element of

[TABLE]

the space of infinite time series that are eventually constant, by extending it constantly. In this section we will see that the appropriate algebraic setting for iterated-sums, combined into the map $\operatorname{ISS}(x)$, is that of a character on the quasi-shuffle Hopf algebra $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star,\delta,\varepsilon,|\cdot|)$ over the semigroup $\mathfrak{A}$ corresponding to the alphabet $A=\{\mathtt{1},\mathtt{2},\dotsc,\mathtt{d}\}$, introduced in Section 2.

The following notation for elements in the time series $x$ is put in place:

[TABLE]

Next we define the corresponding time series

[TABLE]

with increments $(\Delta x)_{n}\coloneqq x_{n}-x_{n-1}\in\mathbb{F}^{d}$, for $n\geq 1$, as entries. The new notation is extended to include all brackets in $\mathfrak{A}$ by defining

[TABLE]

Definition 3.1**.**

The iterated-sums signature of the time series $x$ is the two-parameter family $(\operatorname{ISS}(x)_{n,m}\mid 0\leq n\leq m\in{\N})$ of linear maps from $T(\mathfrak{A})$ to $\mathbb{F}$ such that $\operatorname{ISS}(x)_{n,n}=\varepsilon$, and defined recursively by $\langle e,\operatorname{ISS}(x)_{n,m}\rangle\coloneqq 1$, and for $a_{1}\dotsm a_{p}\in T(\mathfrak{A})$

[TABLE]

Hence, the iterated-sums signature is a word series in $H_{\mathrm{qsh}}^{*}$

[TABLE]

with iterated sums over increments of $x$ as coefficients, defined as

[TABLE]

For example

[TABLE]

We extend this definition to all $n,m\in{\N}$ by setting $\langle w,\operatorname{ISS}(x)_{n,m}\rangle=0$ whenever $m<n$.

Remark 3.2**.**

An easy consequence of this definition is that the coefficient $\langle w,\operatorname{ISS}(x)_{n,m}\rangle$ vanishes whenever $\ell(w)>m-n$.

The proof of the following lemma is straightforward.

Lemma 3.3**.**

Let $x=(x_{n})_{n\geq 0}$ and $x^{\prime}=(x^{\prime}_{n})_{n\geq 0}$ be two time series, and denote by $xx^{\prime}\coloneq(x_{n}x^{\prime}_{n})_{n\geq 0}$. Then the increment of the product $xx^{\prime}$ is given by a generalised Leibniz rule

[TABLE]

More importantly, we have the following:

Theorem 3.4**.**

1. (1)

(Quasi-shuffle identity)* For each $n\leq m$, the map $\operatorname{ISS}(x)_{n,m}:H_{\mathrm{qsh}}\to\mathbb{F}$ is a quasi-shuffle Hopf algebra character.* 2. (2)

(Chen’s property)* For any three $n<n^{\prime}<n^{\prime\prime}\in{\N}$ we have*

[TABLE]

Remark 3.5**.**

1. Observe that point (i) in Theorem 3.4 amounts to a generalisation of the algebra isomorphism defined in (6) to the multidimensional case, i.e., for an alphabet $A=\{\mathtt{1},\ldots,\mathtt{d}\}$. Indeed, defining the map $\Sigma_{d}$ on $H_{\mathrm{qsh}}$

[TABLE]

where $u_{1},\ldots,u_{n}\in\mathfrak{A}$ and for $u=[\mathtt{a}_{1}\cdots\mathtt{a}_{l}]\in\mathfrak{A}$ we have set

[TABLE]

we obtain a quasi-shuffle algebra isomorphism into the algebra of quasi-symmetric functions of level $d$, as introduced by Novelli and Thibon in [42]. For the sake of briefness we only remark that

[TABLE]

2. Specialising to $\mathbb{F}=\mathbb{R}$, Theorem 3.4 matches the corresponding result for the iterated-integrals signature $S(X)$ of a curve of bounded variation in $\R^{B}$, where $B$ is a (possibly countable) alphabet. The iterated-integrals signature is also called Chen’s signature, rough path signature, continuous-time signature or just signature in the literature.

Here, the underlying Hopf algebra is $H_{\shuffle}=(T(B),\shuffle,\delta,\varepsilon)$. Indeed (see for example [23]),

1. (1)

(Shuffle identity)* For fixed $s<t$, $S(X)_{s,t}$ is a character on $H_{\shuffle}$, that is for all $v,w\in H_{\shuffle}$*

[TABLE] 2. (2)

(Chen’s property)* For $s<u<t$*

[TABLE]

Before proving Theorem 3.4 we need the following abstract result, which is a particular case of the setting presented in [42, Section 5.1].

Lemma 3.6**.**

Let $M_{[u_{1}]\dotsm[u_{k}]}(Y)\coloneqq\Sigma_{d}([u_{1}]\dotsm[u_{k}])$ denote the level $d$ monomial quasisymmetric functions defined in (15). Then, the “generating series”

[TABLE]

admits the factorisation

[TABLE]

Let us look at the first few terms in (16):

[TABLE]

Instead of elaborating on this lemma, we refer to reference [42] for details about multivariable generating series. Note, however, that after evaluating $\sigma(Y)$ in $Y_{j}^{[\mathtt{a}]}=\Delta x_{j}^{[\mathtt{a}]}$, we obtain $\operatorname{ISS}(x)$ and the factorisation (16) takes place in the convolution algebra $(T(\mskip-4.0mu(\mathfrak{A})\mskip-4.0mu),\centerdot)$. We further remark that the expansion of the geometric series on the righthand side of the first equality in (16) takes place in $\mathfrak{A}$, which explains the summation over $\mathfrak{A}$ in the second equality.

Remark 3.7**.**

Equality (16) bears resemblance to [32, Definition 4.1] (c.f. also [38, Theorem 32]. We would like to thank Harald Oberhauser (Oxford) for pointing us to these references). At first sight though, only coefficients for words in letters of weight one are considered in the aforementioned reference (e.g. in our notation $[1],[2],\dotsc,[d],[1][1],[1][2],\dots,[1][1][1],$…). Preprocessing the underlying time series through a nonlinear function (i.e. a kernel in the terminology of [32]), one can introduce additional polynomial expressions. But, note that in their setting then nonetheless sums of increments of polynomials appear, whereas in the iterated-sums signature (i.e. in (16) evaluated at $Y_{i}=\Delta x_{i}$) polynomials of increments show up.

The differences between the two approaches may be summarized by saying that increments of polynomials differ from polynomials of increments. Saying this, it is an interesting question how these two approaches could be combined fruitfully. In particular, we hope to investigate the application of kernelization techniques to the iterated-sums signature.

Finally, we would like to mention that the work of Hoffman–Ihara (see Section 5 and [28], as well as [18]) permits to define for any positive integer a linear automorphism of $T(\mathfrak{A})$ which gives rise to a family of “feature maps” interpolating between the iterated-sums signature and the iterated-integrals signature. This relates to a modification of (16) in the spirit of [32, Appendix B]. These new feature maps define characters over Hopf algebras equipped with new quasi-shuffle type products. The corresponding family of linear automorphisms define algebra maps between these quasi-shuffle type products and the quasi-shuffle product (2). We postpone the details of this construction to a follow-up paper, and would like to thank the anonymous referee for hinting at this direction.

Proof 3.8** (Proof of Theorem 3.4).**

1. We need to show that for words $w,w^{\prime}\in T(\mathfrak{A})$

[TABLE]

We use the recursive definition of the quasi-shuffle product (2) and induction on $\ell(w)+\ell(w^{\prime})$, the base case (i.e., $w=e$ or $w^{\prime}=e$) being trivial. If $u,v\in T(\mathfrak{A})$ and $a,b\in\mathfrak{A}$, define the auxiliary time series

[TABLE]

for $0\leq k\leq m-n$, and zero else. Observe that the increments

[TABLE]

By the induction hypothesis we then get

[TABLE]

and similarly

[TABLE]

Also, by a similar argument we also have

[TABLE]

Finally, we summate these relations by using Lemma 3.3 to get

[TABLE]

2. The proof of Chen’s property can be pursued using a pedestrian approach. However, it also follows from Lemma 3.6. Indeed, we may split the product in the factorisation (16) as

[TABLE]

The desired identity follows upon evaluation at $Y_{j}^{[\mathtt{a}]}=\Delta x_{j}^{[\mathtt{a}]}$ as in the previous remark.

We note that the iterated-sums signature, $\operatorname{ISS}(x)_{n,m}$, introduced in this work is similar to the discrete Chen(–Fliess) series defined and studied in [22] in the context of nonlinear control theory.

This section is closed with an intriguing observation. Up to this point it may seem that iterated-sums signatures, $\operatorname{ISS}(x)_{n,m}$, and Chen’s signatures, $S(X)_{s,t}$ (see Remark 3.5), behave in the same way, but as the next example shows this is not at all the case. Recall that $\mathrm{End}_{\mathbb{F}}(H_{\mathrm{qsh}})$, the space of linear maps on $H_{\mathrm{qsh}}$, together with the convolution product $\psi\ast\gamma\coloneqq m_{\star}(\psi\otimes\gamma)\delta,$ is a non-commutative algebra with unit $\iota\coloneqq\eta\circ\varepsilon$, where $\eta:\mathbb{F}\to H_{\mathrm{qsh}}$ is the unit map, $\eta(\lambda):=\lambda e$. Define

[TABLE]

where $J\coloneqq\operatorname{id}-\iota\in\mathrm{End}_{\mathbb{F}}(H_{\mathrm{qsh}})$ is the projection onto the augmentation ideal $T_{+}(\mathfrak{A})$. It is the adjoint of the classical Eulerian Lie idempotent [46], that is, the concatenation logarithm of the identity map, $\log^{\centerdot}(\operatorname{id})$. Observe that the sum (17) terminates when evaluated in homogeneous elements since $J(e)=0$, thus it is well defined for arbitrary elements of $T(\mathfrak{A})$. Then, for any character $c\in T(\mskip-4.0mu(\mathfrak{A})\mskip-4.0mu)$ and word $u\in T(\mathfrak{A})$ we have that

[TABLE]

where $\log^{\centerdot}c\in g$. Indeed, by definition

[TABLE]

In the third equality we used that $c$ is a character. In the second equality the reduced coproduct is applied

[TABLE]

Now, if $x$ is an arbitrary time series, for its iterated-sums signature this means that

[TABLE]

Therefore, the image of the logarithm of iterated-sums signatures only reaches a certain subset of the Lie algebra of infinitesimal characters on $H_{\mathrm{qsh}}$. This is in contrast to Chen’s iterated-integrals signature, for which Chow’s Theorem [19, Theorem 7.28] holds, showing that any character over the shuffle Hopf algebra may be realised as the Chen signature of a piecewise linear path. The implications of this observation will be studied in a forthcoming paper.

Still, the following positive statement on the linear span of iterated-sums signatures holds.

Lemma 3.9**.**

For every $n\geq 1$, $\operatorname{span}_{\mathbb{F}}\{\operatorname{proj}_{\leq n}\operatorname{ISS}(x):x\in(\mathbb{F}^{d})^{\N_{+}}_{c}\}=\operatorname{proj}_{\leq n}H^{*}_{\mathrm{qsh}}$.

Remark 3.10**.**

The corresponding result for iterated-integrals signatures was shown in [13, Lemma 3.4], which is sometimes useful for proving statements about the underlying algebra that are easily verified when tested against signatures.

Proof 3.11**.**

Fix $n\geq 1$ and let $P_{1},\dots,P_{L}$, ordered in some way, be the quasisymmetric monomial functions with degree smaller or equal to $n$.

By [42, Section 5.1] they are independent as elements of the space of formal power series.

This implies that, for some $m\geq 1$ large enough, evaluating at $Y_{m}=(y_{1},\ldots,y_{m},0,0,\ldots)$, the expressions $P_{1}(Y_{m}),\dots,P_{L}(Y_{m})$ are independent as elements of $\mathbb{F}\left[y_{1},\ldots,y_{m}\right]$. Denote

[TABLE]

copies in new variables of $Y_{m}$, that is $P_{i}\left(Y^{(\ell)}_{m}\right)\in R:=\mathbb{F}\left[y^{(1)}_{1},\dots,y^{(1)}_{m},\dots,y^{(L)}_{1},\ldots,y^{(L)}_{m}\right]$. Then, the independence of the $P_{i}$ implies independence, in $R$, of the rows of

[TABLE]

A fortiori, also the columns must be independent in $R$. Hence, the columns must be independent for some realisation of the $Y^{(\ell)}$. This finally implies that we can find $x^{(\ell)}\in(\mathbb{F}^{d})^{m+1},\ell=1,\ldots,L$ such that the columns of

[TABLE]

are independent. Here, as before, we extend the $x^{(\ell)}$ constantly to an element of $(\mathbb{F})^{{\N_{+}}}_{c}$.

4. Invariants

In the previous section, we defined the iterated-sums signature, following the introduction. We now return to our original motivation, and first put the concept of “time warping” in a precise mathematical framework. For each index $n\geq 0$ we define an operator acting on sequences by repeating once the value at time $n$. More precisely, given a time series, $x$, we define $\tau_{n}(x)$ as the time series given by

[TABLE]

Observe that with this definition we have $\tau_{n}(x)_{n}=\tau_{n}(x)_{n+1}=x_{n}$, and the rest of the values are unchanged save for a time shift after time $n$.

Definition 4.1**.**

We call a functional $F:(\mathbb{F}^{d})^{{\N_{+}}}_{c}\to\mathbb{F}$ invariant to time warping if $F\circ\tau_{n}=F$ for all $n\geq 1$.

From applications to data analysis, such as, e.g., moment corrections, we are mostly interested in polynomial invariants, i.e., invariant functionals that can be expressed by considering only polynomial expressions in a time series.

Definition 4.2**.**

We call $F:(\mathbb{F}^{d})^{{\N_{+}}}_{c}\to\mathbb{F}$ polynomial, if for all $N\geq 1$, $F(x_{0},\dotsc,x_{N},0,0,\dotsc)$ is a polynomial in the $x_{i}$, and the polynomial degree is uniformly bounded in $N$.

From the factorisation (16) in Lemma 3.6 it follows that for any word $w\in T(\mathfrak{A})$ the coefficient $\langle w,\operatorname{ISS}(x)\rangle$ is a polynomial invariant in this sense. It turns out these are all the polynomial invariants, if we additionally demand invariance with respect to space translation of the entire series.

Lemma 4.3**.**

Let $F$ be polynomial, invariant to both time warping and space translations. Then: $F$ is realised as a quasisymmetric function.

Proof 4.4**.**

We do the one-dimensional case, $d=1$, to avoid notational clutter. By translation invariance, for any $N\geq 1$,

[TABLE]

Now, by assumption, this is a polynomial in $x_{1}-x_{0},x_{2}-x_{0},\ldots,x_{N}-x_{0},$ hence it is a (different) polynomial in $x_{1}-x_{0},x_{2}-x_{1},x_{3}-x_{2},\ldots,x_{N}-x_{N-1}$. Therefore, $F$ can be realised as a formal power series of bounded degree: there is $\hat{F}\in\mathbb{F}\langle Y_{1},Y_{2},\dots\rangle$ of bounded degree such that for $x\in(\mathbb{F}^{d})^{{\N_{+}}}_{c}$ we have that $F(x)=\hat{F}(\Delta x)$.

It remains to show that $\hat{F}$ is quasisymmetric. Let $n\geq 1$, $i_{1}<\cdots<i_{n}$ and $\alpha_{1},\dots,\alpha_{n}\geq 1$. We show that the coefficient of the monomial $Y_{i_{1}}^{\alpha_{1}}\cdots Y_{i_{n}}^{\alpha_{n}}$ in $\hat{F}$ is equal to the one of $Y_{1}^{\alpha_{1}}\cdots Y_{n}^{\alpha_{n}}$.

Indeed: by using repeatedly the invariance to time warping, we get that for all $x\in\R^{n}$,

[TABLE]

Hence, both sides coincide as polynomials. So that the coefficient of $Y_{i_{1}}^{\alpha_{1}}\cdots Y_{i_{n}}^{\alpha_{n}}$ and $Y_{1}^{\alpha_{1}}\dots Y_{n}^{\alpha_{n}}$ must coincide. This finishes the proof.

5. Hoffman’s isomorphism and signatures

In this section we relate the iterated-sums signature of a time series with the usual iterated-integrals signature of the piecewise linear interpolation of an associated infinite dimensional time series.

Starting again with the extended alphabet $\mathfrak{A}$, we build the tensor algebra $T(\mathfrak{A})$ and define the shuffle product $\shuffle:T(\mathfrak{A})\otimes T(\mathfrak{A})\to T(\mathfrak{A})$ inductively by

[TABLE]

Recall Hoffman’s isomorphism [27] defined in Theorem 2.6, which shows that $H_{\shuffle}=(T(\mathfrak{A}),\shuffle,\delta)$ and $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star,\delta)$ are isomorphic as Hopf algebras. Next we compute explicitly the image by the iterated-integrals signature $S$ of a linear path.

The following lemma is an immediate extension of [19, Example 7.21] to a countable index set.

Lemma 5.1**.**

Consider a countable set $B$ and let $z_{t}=z_{0}+at$ for some $z_{0},a\in\mathbb{R}^{B}$ and all $t\in[0,1]$. Then for $w=w_{1}\cdots w_{n}\in T(B)$

[TABLE]

At the level of the tensor algebra this simply means that $S(z)_{s,t}=\exp^{\centerdot}((t-s)a)$. An analogue of this result holds for discrete signatures, which follows from Lemma 3.6, i.e., Chen’s property.

Lemma 5.2**.**

Let $x=(0,v,v,\dotsc)$ be a time series having a single non-zero increment $v=(v^{[\mathtt{1}]},\dotsc,v^{[\mathtt{d}]})\in\mathbb{R}^{d}$. Then

[TABLE]

Now we look for a relation between the iterated-integrals signature and the iterated-sums signature. For this, let $x=(0,x_{1},x_{2},\dotsc)$ be a time series and consider the (infinite dimensional!) path $X=(X^{a}:a\in\mathfrak{A})$ where, for $a=[\mathtt{1}^{k_{1}}\dotsm\mathtt{d}^{k_{d}}]\in\mathfrak{A}$, the component path $X^{a}$ is the linear interpolation of the time series

[TABLE]

Theorem 5.3**.**

We have $\langle\Phi_{\mathrm{H}}(w),\operatorname{ISS}(x)\rangle=\langle w,S(X)\rangle$.

Remark 5.4**.**

We note that the iterated-integrals signature of the $d$-dimensional path consisting in the piecewise linear interpolation of $x$ is not enough to obtain $\operatorname{ISS}(x)$. Instead, the theorem shows that the iterated-integrals signature of the piecewise linear interpolation of the infinite dimensional time series (18) is sufficient.

Proof 5.5**.**

Without loss of generality let the interpolation of (18) happen at the time points $0,1,2,\dots,N$. Then, by Chen’s property,

[TABLE]

We first investigate what happens for a single time step. Let a word $w=[a_{1}]\dotsm[a_{p}]\in T(\mathfrak{A})$ be given, and write $a_{i}=[\mathtt{1}^{k_{1}^{i}}\dotsm\mathtt{d}^{k_{d}^{i}}]\in\mathfrak{A}$, $i=1,\dotsc,p$. According to Lemma 5.1,

[TABLE]

where $K_{\mathtt{m}}\coloneqq k_{\mathtt{m}}^{1}+\dotsb+k_{\mathtt{m}}^{p}$. In other words, $K_{\mathtt{m}}$ is the number of times the letter $\mathtt{m}\in\{\mathtt{1},\dotsc,\mathtt{d}\}$ is repeated in $w$.

Now the only term in $\Phi_{\mathrm{H}}(w)$ containing a single letter is $\frac{1}{p!}[\mathtt{1}^{K_{\mathtt{1}}}\dotsm\mathtt{d}^{K_{\mathtt{d}}}]$, i.e., the full “contraction”. Then, by Lemma 5.2,

[TABLE]

Therefore, we have shown the claim for a single time step.

Now, since $\Phi_{\mathrm{H}}$ is a Hopf algebra map, the statement of the theorem is equivalent to showing that $\Phi_{\mathrm{H}}^{*}(\operatorname{ISS}(x))=S(X)$, where $\Phi_{\mathrm{H}}^{*}$ is the adjoint of Hoffman’s isomorphism. Since $\Phi_{\mathrm{H}}^{*}$ is an algebra morphism, we calculate

[TABLE]

So the result is valid for the full signature.

Finally, we show a consistency result.

Proposition 5.6**.**

Let $X\colon[0,1]\to\R^{d}$ be a continuous path of finite variation, meaning that

[TABLE]

where the supremum is taken over all partitions $\pi$ of $[0,1]$.

Given such a partition $\pi=\{t_{0}=0<t_{1}<\dotsb<t_{N-1}<t_{N}=1\}$, define $x(\pi)$ by $x(\pi)_{j}=X_{t_{j}}$. Then

[TABLE]

Proof 5.7**.**

We use induction on the length $\ell(w)$. If $\ell(w)=1$ and $w=\mathtt{i}\in A$, then

[TABLE]

which is independent of $\pi$. If, on the other hand, $w=a\in\mathfrak{A}\setminus A$ then $a=[\mathtt{1}^{k_{1}}\dotsm\mathtt{d}^{k_{d}}]$ with $k_{1}+\dotsb+k_{d}\geq 2$. Therefore

[TABLE]

which vanishes in the limit since $X$ is uniformly continuous on $[0,1]$.

Now suppose $w=w^{\prime}a$ for some $w\in T(\mathfrak{A})$ and $a\in\mathfrak{A}$. We have 3 cases

1. (1)

$w^{\prime}\in T(\mathfrak{A}\setminus A)$: in this case, no matter what $a$ is, we have

[TABLE]

as $\lvert\pi\rvert\to 0$, by the induction hypothesis. 2. (2)

$w^{\prime}\in T(A)$* and $a\in\mathfrak{A}\setminus A$: the same argument as before gives that the corresponding entry in $\operatorname{ISS}(x(\pi))$ vanishes in the limit.* 3. (3)

$w^{\prime}\in T(A)$* and $a\in A$: again by definition we have*

[TABLE]

which converges to the Young (or Riemann–Stieltjes) integral

[TABLE]

Therefore, we have that

[TABLE]

if $w\in T(A)$, and vanishes otherwise.

5.1. The area operation

It is well known that for the iterated-integrals signature certain linear combinations of the entries have a precise geometric interpretation. Indeed, for any $\mathtt{i},\mathtt{j}\in A$

[TABLE]

represents (two times) the signed area (or Lévy area) between the curves $u\mapsto X_{u}^{\mathtt{i}}$ and $u\mapsto X_{u}^{\mathtt{j}}$ for $u\in[s,t]$, and the cord between the points $(X_{s}^{\mathtt{i}},X_{s}^{\mathtt{j}})$ and $(X_{t}^{\mathtt{i}},X_{t}^{\mathtt{j}})$.

We abstract this operation to the shuffle algebra by using the notion of half-shuffles introduced in Section 2.1. In fact, one verifies that at this level the area operation may be represented in terms of half-shuffle operations as

[TABLE]

so that in particular $\operatorname{Area}(X^{\mathtt{i}},X^{\mathtt{j}})_{s,t}=\langle\operatorname{area}(\mathtt{i},\mathtt{j}),S(X)_{s,t}\rangle$.

We extend this by defining area operations on $H_{\shuffle}=(T(\mathfrak{A}),\shuffle,\delta)$ and $H_{\mathrm{qsh}}=(T(\mathfrak{A}),\star,\delta)$.

Definition 5.8**.**

The area map $\operatorname{area}\colon H_{\shuffle}\otimes H_{\shuffle}\to H_{\shuffle}$ is defined by

[TABLE]

Next, the discrete analogue is given in terms of the first half-shuffle product in (7).

Definition 5.9** (Discrete area).**

The discrete area map $\operatorname{\mathtt{area}}\colon H_{\mathrm{qsh}}\otimes H_{\mathrm{qsh}}\to H_{\mathrm{qsh}}$ is defined by

[TABLE]

We compare the two areas by considering the words $u=[\mathtt{3}]$ and $v=[\mathtt{4}][\mathtt{12}]$. Then

[TABLE]

as follows from Example 8.

Both $\operatorname{area}$ and $\operatorname{\mathtt{area}}$ can be iterated. We now make this precise: define $\mathsf{D}_{1}=D_{1}\coloneqq\mathbb{F}\,\mathfrak{A}$, the vector space spanned by the set $\mathfrak{A}$. Then, inductively define vector spaces

[TABLE]

We finally set

[TABLE]

Neither the $\operatorname{area}$ nor the discrete $\operatorname{\mathtt{area}}$ operations are associative. One can show, however, that $\operatorname{area}$ satisfies a fourth-order relation, known as tortkara, introduced by Dzhumadil’daev in the 2007 paper [14]. In [15] the image of iterated applications of the area map is characterised. (Compare also [45, Theorem 28]).

Theorem 5.10** ([15, Theorem 2.1]).**

The space $D$ is spanned by the set

[TABLE]

From Lemma 2.7 and Lemma 2.8 we deduce the following morphism property of Hoffman’s isomorphism with respect to $\operatorname{area}$ and $\operatorname{\mathtt{area}}$.

Theorem 5.11**.**

$\Phi_{\mathrm{H}}\colon D\to\mathsf{D}$* is a tortkara morphism, i.e., for $\varphi,\psi\in D$*

[TABLE]

Remark 5.12**.**

1. Note that $\Phi_{\mathrm{H}}$ is not a (quasi-)half-shuffle morphism. Only the anti-symmetrisation to $\operatorname{area}$ respectively $\operatorname{\mathtt{area}}$ is nicely compatible with it.

2. The set $D$ (the set of “areas-of-areas”) is known to generate $T(\mathfrak{A})$ as a shuffle-algebra, see [12]. Applied to iterated-integral signatures this means that all their information is already contained in areas-of-areas. The area operation $(X,Y)\mapsto\operatorname{Area}(X,Y)$ has an immediate geometric interpretation, whereas the operation of integration $(X,Y)\mapsto\int X\,\mathrm{d}Y$.111The authors would be hard-pressed to explain the latter to a non-mathematician, whereas the former can be explained by a simple drawing. Moreover, the area operation is related to antisymmetrised lead-lag correlation in time series analysis, see [13, Section 3.2]. We refer to [12, Section 6] for more applications.

Proof 5.13**.**

By Dzumadil’daev’s theorem (Theorem 5.10) it suffices to prove the claim for the case when $\varphi=u([a][b]-[b][a])$ and $\psi=v([c][d]-[d][c])$. We first observe that in this case the $\operatorname{area}$ operation can be written more explicitly:

[TABLE]

Each of these terms can be further expanded into three terms. For example, the first one equals

[TABLE]

In total there are 12 terms, the remaining 9 terms are

[TABLE]

For each of these terms we can find exactly one other term such that their sum is of the form $w([x][y]-[y][x])$, for $[x],[y]\in\{[a],[b],[c],[d]\}$, and thus by Lemma 2.8 the image of this sum has the form $\Phi_{\mathrm{H}}(w[x])[y]-\Phi_{\mathrm{H}}(w[y])[x]$. To summarise, the image $\Phi_{\mathrm{H}}(\operatorname{area}(\varphi,\psi))$ is a linear combination of 6 terms, each of them having the form $\Phi_{\mathrm{H}}(w[x])[y]-\Phi_{\mathrm{H}}(w[y])[x]$. Now, if we pick any $[x]\in\{[a],[b],[c],[d]\}$ there are exactly three terms containing $[x]$ as the last letter. For example, for $[a]$ these terms are

[TABLE]

where the last identity is easy to check using that $\psi=v([c][d]-[d][c])$. Applying a similar argument to all letters we see that

[TABLE]

6. Conclusion

In this work we have

• •

introduced a new set of features for multidimensional time series consisting in iterated sums (Section 3);

• •

shown that these features are invariant to time warping and that these in fact are all the (polynomial) invariants in this sense (Section 4);

• •

described a Hopf algebraic framework to compute these features (Section 2);

• •

shown how this setting mirrors the one of iterated-integrals in some aspects and differs in others (Section 2).

There are several possible generalisations of our work.

• •

Let $f,g:\mathbb{F}\to\mathbb{F}$ be such that $f(0)=g(0)=0$. Then iterated-sums of the form

[TABLE]

are also invariant to time warping (and analogously for higher order iterated-sums). These are, in general, not polynomial in the time series anymore, but might still be relevant for certain applications. For smooth $f,g$ this should be related to the expansion of nonlinear functionals on stochastic word series [11], but the non-smooth case (for example $f(x)=x$, $g(x)=|x|$) is particularly interesting.

• •

Multi-parameter data. An object of interest are for example “images” $I:[0,N]\times[0,N]\to\R$ and the time warping invariance becomes an invariance to stretching of the image.

We are also interested in exploring the possible applications of these invariants in data science.

• •

Retrieval of similar time series, invariant to time warping: see [47] (and references therein), where it is stated that “the time warping distance …does not lead to any natural features”. The invariants presented in our work should provide those missing features, but a mathematical rigorous proof of this statement is left for future work.

• •

Statistical inference in problems involving unknown time warping, as in Example 1.2.

• •

Time series clustering: the features of this work can be used to cluster time series according to their “shape”, i.e., independent of time warping. Sometimes a “prototype” for each cluster is looked after, see for example [43]. In this case - as in the previous point - reconstruction of a time series from an (averaged) iterated-sums signature would be necessary. A detailed study of this ostensibly hard problem is left for future research.

We close with some open questions. At the end of Section 3 we showed that an equivalent of Chow’s theorem does not hold for the iterated-sums signature $\operatorname{ISS}(x)$.

• •

Can we understand $\{\operatorname{ISS}(x):x\in(\mathbb{F}^{d})^{\N_{+}}_{c}\}$ as a semi-algebraic set? (Compare [3] for the investigation of the image of iterated-integrals signatures as algebraic sets.)

• •

For $x\in(\mathbb{F}^{d})^{N}$ denote by $\overleftarrow{x}$ the time series run backwards. Then (as might surprise readers familiar with Chen’s signature) $\operatorname{ISS}(\overleftarrow{x})\centerdot\operatorname{ISS}(x)\neq\varepsilon$. What are the implications?

• •

The lead-lag procedure of [17] lifts a discrete time series of dimension $d$ to a piecewise smooth curve of dimension $2d$. Since the resulting iterated-integrals signature is invariant to time warping as well as space translations, and is polynomial in the original time series, by Lemma 4.3 it must be contained in the iterated-integrals signature $\operatorname{ISS}(x)$. Conversely, is the signature of the resulting $2d$ curve enough to recover the iterated-sum signature? This would give a finite dimensional smooth curve whose iterated-integrals signature contains the invariants presented in this paper (compare Theorem 5.3 for an infinite dimensional smooth curve doing the job).

Acknowledgments The authors would like to thank the referee for valuable remarks and comments, in particular with respect to reference [32]. N.T. kindly acknowledges support from the European Research Consortium for Informatics and Mathematics through post-doctral fellowship ERCIM 2018-10 and from the Excelence Cluster MATH+ EF1.