The Parameterized Position Heap of a Trie

TL;DR

This paper introduces a parameterized position heap data structure for tries that efficiently supports p-matching queries, enabling effective pattern matching with bijections on disjoint alphabets.

Contribution

It proposes a space-efficient data structure and an algorithm for constructing it, specifically designed for parameterized pattern matching on tries.

Findings

Supports p-matching queries in O(m log (σ + π) + m π + pocc) time.

Uses O(N) space for the data structure.

Construction algorithm runs in O(N (σ + π)) time.

Abstract

Let $\Sigma$ and $\Pi$ be disjoint alphabets of respective size $\sigma$ and $\pi$. Two strings over $\Sigma \cup \Pi$ of equal length are said to parameterized match (p-match) if there is a bijection $f:\Sigma \cup \Pi \rightarrow \Sigma \cup \Pi$ such that (1) $f$ is identity on $\Sigma$ and (2) $f$ maps the characters of one string to those of the other string so that the two strings become identical. We consider the p-matching problem on a (reversed) trie $\mathcal{T}$ and a string pattern $P$ such that every path that p-matches $P$ has to be reported. Let $N$ be the size of the given trie $\mathcal{T}$. In this paper, we propose the parameterized position heap for $\mathcal{T}$ that occupies $O(N)$ space and supports p-matching queries in $O(m \log (\sigma + \pi) + m \pi + \mathit{pocc}))$ time, where $m$ is the length of a query pattern $P$ and $\mathit{pocc}$ is the number of paths in $\mathcal{T}$ to report. We also present an algorithm which constructs the parameterized position heap for a given trie $\mathcal{T}$ in $O(N (\sigma + \pi))$ time and working space.

Equations

$$\mathsf{SHT}(\mathcal{S})^{i}=\begin{cases}(\{\varepsilon\},\emptyset)&\mbox{if $i=1$},\\ (V_{i-1}\cup\{u_{i}\},E_{i-1}\cup\{(v_{i},a,u_{i})\})&\mbox{if $2\leq i\leq k$},\end{cases}$$

$$\mathsf{rsl}(a,v)=\begin{cases}\mathsf{spe}(av)&\mbox{if $\mathsf{spe}(av)$ is represented by $\mathsf{PPH}(\mathcal{T})$},\\ \mbox{undefined}&\mbox{otherwise}.\end{cases}$$

$$\mathsf{rsl}(a,h_k)$$

$$\mathsf{rsl}(a,h_k)=\mathsf{spe}(a\cdot \mathsf{spe}(w_j[\mathrm{1..}\ell -1]))=\mathsf{spe}(a\cdot w_j[\mathrm{1..}\ell -1])=\mathsf{spe}(w_i[\mathrm{1..}\ell ]).$$

Full text

1 Introduction

The parameterized matching problem (p-matching problem), first introduced by Baker [2], is a variant of pattern matching which looks for substrings of a text that has “the same structure” as a given pattern. More formally, we consider a parameterized string (p-string) that can contain static characters from an alphabet $\Sigma$ and parameter characters from another alphabet $\Pi$. Two equal length p-strings $x$ and $y$ over the alphabet $\Sigma\cup\Pi$ are said to parameterized match (p-match) if $x$ can be transformed to $y$ (and vice versa) by applying a bijection which renames the parameter characters. The p-matching problem is, given a text p-string $w$ and pattern p-string $p$, to report the occurrences of substrings of $w$ that p-match $p$. Studying the p-matching problem is well motivated by plagiarism detection, software maintenance, and RNA structural pattern matching [2, 15]. We refer readers to [11] for detailed descriptions about these motivations.

Baker [2] proposed an indexing data structure for the p-matching problem, called the parameterized suffix tree (p-suffix tree). The p-suffix tree supports p-matching queries in $O(m\log(\sigma+\pi)+\mathit{pocc})$ time, where $m$ is the length of pattern $p$, $\sigma$ and $\pi$ are respectively the sizes of the alphabets $\Sigma$ and $\Pi$, and $\mathit{pocc}$ is the number of occurrences to report [1]. She also showed an algorithm that builds the p-suffix tree for a given text $S$ of length $n$ in $O(n(\pi+\log\sigma))$ time with $O(n)$ space [2]. Later, Kosaraju [8] proposed an algorithm to build the p-suffix tree in $O(n\log(\sigma+\pi))$ time111The original claimed time bounds in Kosaraju [8] and in Shibuya [15] are $O(n(\log\sigma+\log\pi))$. However, assuming by symmetry that $\sigma\geq\pi$, we have $\log\sigma+\log\pi=\log(\sigma\pi)\leq\log\sigma^{2}=2\log\sigma=O(\log\sigma)$ and $\log(\sigma+\pi)\leq\log(2\sigma)=\log 2+\log\sigma=O(\log\sigma)$. with $O(n)$ space. Their algorithms are both based on McCreight’s suffix tree construction algorithm [10], and hence are offline (namely, the whole text has to be known beforehand). Shibuya [15] gave an left-to-right online algorithm that builds the p-suffix tree in $O(n\log(\sigma+\pi))$ time with $O(n)$ space. His algorithm is based on Ukkonen’s suffix tree construction algorithm [16] which scans the input text from left to right.

Diptarama et al. [5] proposed a new indexing structure called the parameterized position heap (p-position heap). They showed how to construct the p-position heap of a given p-string $S$ of length $n$ in $O(n\log(\sigma+\pi))$ time with $O(n)$ space in a left-to-right online manner. Their algorithm is based on Kucherov’s position heap construction algorithm [9] which scans the input text from left to right. Recently, Fujisato et al. [7] presented another variant of the p-position heap that can be constructed in a right-to-left online manner, in $O(n\log(\sigma+\pi))$ time with $O(n)$ space. This algorithm is based on Ehrenfeucht et al.’s algorithm [6] which scans the input text from right to left. Both versions of p-positions heaps support p-matching queries in $O(m\log(\sigma+\pi)+m\pi+\mathit{pocc})$ time.

This paper deals with indexing on multiple texts; in particular, we consider the case where those multiple texts are represented by a trie. It should be noted that our trie is a so-called common suffix trie (CS trie) where the common suffixes of the texts are merged and the edges are reversed (namely, each text is represented by a path from a leaf to the root). See also Figure 1 for an example of a CS trie. There are two merits in representing multiple texts by a CS trie: Let $N$ be the size of the CS trie of the multiple strings of total length $Z$. (1) $N$ can be as small as $\Theta(\sqrt{Z})$ when the multiple texts share a lot of common long suffixes. (2) The number of distinct suffixes of the texts is equal to the number of the nodes in the CS trie, namely $N$. On the other hand, this is not the case with the ordinal common prefix trie (CP trie), namely, the number of distinct suffixes in the CP trie can be super-linear in the number of its nodes. Since most, if not all, indexing structures require space that is dependent of the number of distinct suffixes, the CS trie is a more space economical representation for indexing than its CP trie counterpart.

Let $N$ be the size of a given CS trie. Due to Property (1) above, it is significant to construct an indexing structure directly from the CS trie. Note that if we expand all texts from the CS trie, then the total string length can blow up to $O(N^{2})$. Breslauer [3] introduced the suffix tree for a CS trie which occupies $O(N)$ space, and proposed an algorithm which constructs it in $O(N\sigma)$ time and working space. Using the suffix tree of a CS trie, one can report all paths of the CS trie that exactly matches with a given pattern of length $m$ in $O(m\log\sigma+\mathit{occ})$ time, where $\mathit{occ}$ is the number of such paths to report. Shibuya [14] gave an optimal $O(N)$-time construction for the suffix tree for a CS trie in the case of integer alphabets of size $N^{O(1)}$. Nakashima et al. [12] proposed the position heap for a CS trie, which can be built in $O(N\sigma)$ time and working space and supports exact pattern matching in $O(m\log\sigma+\mathit{occ})$ time. Later, an optimal $O(N)$-time construction algorithm for the position heap for a CS trie in the case of integer alphabets of size $N^{O(1)}$ was presented [13].

In this paper, we propose the parameterized position heap for a CS trie $\mathcal{T}$, denoted by $\mathsf{PPH}(\mathcal{T})$, which is the first indexing structure for p-matching on a trie. We show that $\mathsf{PPH}(\mathcal{T})$ occupies $O(N)$ space, supports p-matching queries in $O(m\log(\sigma+\pi)+m\pi+\mathit{pocc})$ time, and can be constructed in $O(N(\sigma+\pi))$ time and working space. Hence, we achieve optimal pattern matching and construction in the case of constant-sized alphabets. The proposed construction algorithm is fairly simple, yet uses a non-trivial idea that converts a given CS trie into a smaller trie based on the p-matching equivalence. The simplicity of our construction algorithm comes from the fact that each string stored in (p-)position heaps is represented by an explicit node, while it is not the case with (p-)suffix trees. This nice property makes it easier and natural to adopt the approaches by Brealauer [3] and by Fujisato et al. [7] that use reversed suffix links in order to process the texts from left to right. We also remark that all existing p-suffix tree construction algorithms [2, 8, 15] in the case of a single text require somewhat involved data structures due to non-monotonicity of parameterized suffix links [1, 2], but our p-position heap does not need such a data structure even in the case of CS tries (this will also be discussed in the concluding section).

2 Preliminaries

Let $\Sigma$ and $\Pi$ be disjoint ordered sets called a static alphabet and a parameterized alphabet, respectively. Let $\sigma=|\Sigma|$ and $\pi=|\Pi|$. An element of $\Sigma$ is called an s-character, and that of $\Pi$ is called a p-character. In the sequel, both an s-character and a p-character are sometimes simply called a character. An element of $\Sigma^{*}$ is called a string, and an element of $(\Sigma\cup\Pi)^{*}$ is called a p-string. The length of a (p-)string $w$ is the number of characters contained in $w$. The empty string $\varepsilon$ is a string of length 0, namely, $|\varepsilon|=0$. For a (p-)string $w=xyz$, $x$, $y$ and $z$ are called a prefix, substring, and suffix of $w$, respectively. The set of prefixes of a (p-)string $w$ is denoted by $\mathsf{Prefix}(w)$. The $i$-th character of a (p-)string $w$ is denoted by $w[i]$ for $1\leq i\leq|w|$, and the substring of a (p-)string $w$ that begins at position $i$ and ends at position $j$ is denoted by $w[i..j]$ for $1\leq i\leq j\leq|w|$. For convenience, let $w[i..j]=\varepsilon$ if $j<i$. Also, let $w[i..]=w[i..|w|]$ for any $1\leq i\leq|w|$. For any (p-)string $w$, let $w^{R}$ denote the reversed string of $w$, i.e., $w^{R}=w[|w|]\cdots w[1]$.

Two p-strings $x$ and $y$ of length $k$ each are said to parameterized match (p-match) iff there is a bijection $f$ on $\Sigma\cup\Pi$ such that $f(a)=a$ for any $a\in\Sigma$ and $f(x[i])=y[i]$ for all $1\leq i\leq k$. For instance, let $\Sigma=\{\mathtt{a},\mathtt{b}\}$ and $\Pi=\{\mathtt{x},\mathtt{y},\mathtt{z}\}$, and consider two p-strings $x=\mathtt{axbzzayx}$ and $y=\mathtt{azbyyaxz}$. These two strings p-match, since $x$ can be transformed to $y$ by applying a renaming bijection $f$ such that $f(\mathtt{a})=\mathtt{a}$, $f(\mathtt{b})=\mathtt{b}$, $f(\mathtt{x})=\mathtt{z}$, $f(\mathtt{y})=\mathtt{x}$, and $f(\mathtt{z})=\mathtt{y}$ to the characters in $x$. We write $x\approx y$ iff two p-strings $x$ and $y$ p-match. It is clear that $\approx$ is an equivalence relation on p-strings over $\Sigma\cup\Pi$. We denote by $[x]$ the equivalence class for p-string $x$ w.r.t. $\approx$. The representative of $[x]$ is the lexicographically smallest p-string in $[x]$, which is denoted by $\mathsf{spe}(x)$. It is clear that two p-strings $x$ and $y$ p-match iff $\mathsf{spe}(x)=\mathsf{spe}(y)$. In the running example, $\mathsf{spe}(\mathtt{axbzzayx})=\mathsf{spe}(\mathtt{azbyyaxz})=\mathtt{axbyyazx}$

A common suffix trie (CS trie) $\mathcal{T}$ is a reversed trie such that (1) each edge is directed towards the root, (2) each edge is labeled with a character from $\Sigma\cup\Pi$, and (3) the labels of the in-coming edges to each node are mutually distinct. Each node of the trie represents the (p-)string obtained by concatenating the labels on the path from the node to the root. An example of a CS trie is illustrated in Figure 1. $\mathsf{CST}(W)$ denotes the CS trie which represents a set $W$ of (p-)strings.

3 Parameterized position heap of a common suffix trie

In this section, we introduce the parameterized pattern matching (p-matching) problem on a common suffix trie that represents a set of p-strings, and propose an indexing data structure called a parameterized position heap of a trie.

3.1 p-matching problem on a common suffix trie

We introduce the p-matching problem on a common suffix trie $\mathcal{T}$ and a pattern $p$. We will say that a node $v$ in a common suffix trie p-matches with a pattern p-string $p$ if the prefix of length $|p|$ of the p-string represented by $v$ and $p$ p-match. In this problem, we preprocess a given common suffix trie $\mathcal{T}$ so that later, given a query pattern $p$, we can quickly answer every node $v$ of $\mathcal{T}$ whose prefix of length $|p|$ and $p$ p-match. For the common suffix trie in Figure 1, when given query pattern $P=\mathtt{azy}$, then we answer the nodes $17$ and $23$.

Let $W_{\mathcal{T}}$ be the set of all p-strings represented by nodes of $\mathcal{T}$. By the definition of the common suffix trie, there may exist two or more nodes which represent different p-strings, but p-match. We consider the common suffix trie which merges such nodes into the same node by using the representative of the parameterized equivalent class of these strings. We define the set $\mathsf{pcs}(\mathcal{T})$ of p-strings as follows: $\mathsf{pcs}(\mathcal{T})=\{\mathsf{spe}(w^{R})^{R}\mid w\in W_{\mathcal{T}}\}$. Then, the reversed trie which we want to consider is $\mathsf{CST}(\mathsf{pcs}(\mathcal{T}))$. We refer to this reversed trie as the parameterized-common suffix trie of $\mathcal{T}$, and denote it by $\mathsf{pCST}(\mathcal{T})$ (i.e., $\mathsf{pCST}(\mathcal{T})=\mathsf{CST}(\mathsf{pcs}(\mathcal{T}))$). Each node of $\mathsf{pCST}(\mathcal{T})$ stores pointers to its corresponding node(s) of $\mathcal{T}$. Then, by solving the p-matching problem on $\mathsf{pCST}(\mathcal{T})$, we can immediately answering p-matching queries on $\mathcal{T}$. Figure 2 shows an example of $\mathsf{pCST}(\mathcal{T})$. In the rest of this paper, $N$ denotes the number of nodes of $\mathcal{T}$ and $N_{p}$ denotes the number of nodes of $\mathsf{pCST}(\mathcal{T})$. Note that $N\geq N_{p}$ always holds.

3.2 Parameterized position heap of a common suffix trie

Let $\mathcal{S}=\langle s_{1},\ldots,s_{k}\rangle$ be a sequence of strings such that for any $1<i\leq k$, $s_{i}\not\in\mathsf{Prefix}(s_{j})$ for any $1\leq j<i$.

Definition 1* (Sequence hash trees [4]).*

The sequence hash tree of a sequence $\mathcal{S}=\langle s_{1},\ldots,s_{k}\rangle$ of strings, denoted $\mathsf{SHT}(\mathcal{S})=\mathsf{SHT}(\mathcal{S})^{k}$, is a trie structure that is recursively defined as follows: Let $\mathsf{SHT}(\mathcal{S})^{i}=(V_{i},E_{i})$. Then

[TABLE]

where $v_{i}$ is the longest prefix of $s_{i}$ which satisfies $v_{i}\in V_{i-1}$, $a=s_{i}[|v_{i}|+1]$, and $u_{i}$ is the shortest prefix of $s_{i}$ which satisfies $u_{i}\notin V_{i-1}$.

Note that since we have assumed that each $s_{i}\in\mathcal{S}$ is not a prefix of $s_{j}$ for any $1\leq j<i$, the new node $u_{i}$ and new edge $(v_{i},a,u_{i})$ always exist for each $1\leq i\leq k$. Clearly $\mathsf{SHT}(\mathcal{S})$ contains $k$ nodes (including the root).

Let $\mathcal{W}_{\mathcal{T}}=\langle\mathsf{spe}(w_{1}),\ldots,\mathsf{spe}(w_{N_{p}})\rangle$ be a sequence of p-strings such that $\{w_{1},\ldots,w_{N_{p}}\}=\mathsf{pcs}(\mathcal{T})$ and $|w_{i}|\leq|w_{i+1}|$ for any $1\leq i\leq N_{p}-1$. $\mathcal{W}_{\mathcal{T}}(i)$ denote the sequence $\langle\mathsf{spe}(w_{1}),\ldots,\mathsf{spe}(w_{i})\rangle$ for any $1\leq i\leq N_{p}$, and $\mathsf{pCST}(\mathcal{T})^{i}$ denote the common suffix trie of $\{\mathsf{spe}(w_{1}),\ldots,\mathsf{spe}(w_{i})\}$, namely, $\mathsf{pCST}(\mathcal{T})^{i}=\mathsf{CST}(\{\mathsf{spe}(w_{1}),\ldots,\mathsf{spe}(w_{i})\})$. The node of $\mathsf{pCST}(\mathcal{T})$ which represents $w_{i}$ is denoted by $\mathit{c}_{i}$. Then, our indexing data structure is defined as follows.

Definition 2* (Parameterized positions heaps of a CST).*

The parameterized position heap (p-position heap) for a common suffix trie $\mathcal{T}$, denoted by $\mathsf{PPH}(\mathcal{T})$, is the sequence hash tree of $\mathcal{W}_{\mathcal{T}}$ i.e., $\mathsf{PPH}(\mathcal{T})=\mathsf{SHT}(\mathcal{W}_{\mathcal{T}})$.

Let $\mathsf{PPH}(\mathcal{T})^{i}=\mathsf{SHT}(\mathcal{W}_{\mathcal{T}}(i))$ for any $1\leq i\leq N_{p}$ (i.e., $\mathsf{PPH}(\mathcal{T})^{N_{p}}=\mathsf{PPH}(\mathcal{T})$). The following lemma shows the exact size of $\mathsf{PPH}(\mathcal{T})$.

Lemma 1**.**

For any common suffix trie $\mathcal{T}$ such that the size of $\mathsf{pCST}(\mathcal{T})$ is $N_{p}$, $\mathsf{PPH}(\mathcal{T})$ consists of exactly $N_{p}$ nodes. Also, there is a one-to-one correspondence between the nodes of $\mathsf{pCST}(\mathcal{T})$ and the nodes of $\mathsf{PPH}(\mathcal{T})$.

Proof.

Initially, $\mathsf{PPH}(\mathcal{T})^{1}$ consists only of the root that represents $\varepsilon$ since $w_{1}=\varepsilon$. Let $i$ be an integer in $[1..N_{p}]$. Since $w_{i}$ does not p-match with $w_{j}$ and $|\mathsf{spe}(w_{i})|\geq|\mathsf{spe}(w_{j})|$ for any $1\leq j<i$, there is a prefix of $\mathsf{spe}(w_{i})$ that is not represented by any node of $\mathsf{PPH}(\mathcal{T})^{i-1}$. Therefore, when we construct $\mathsf{PPH}(\mathcal{T})^{i}$ from $\mathsf{PPH}(\mathcal{T})^{i-1}$, then exactly one node is inserted, which corresponds to the node representing $w_{i}$. ∎

Let $\mathit{h}_{i}$ be the node of $\mathsf{PPH}(\mathcal{T})$ which corresponds to $w_{i}$. For any p-string $p\in(\Sigma\cup\Pi)^{+}$, we say that $p$ is represented by $\mathsf{PPH}(\mathcal{T})$ iff $\mathsf{PPH}(\mathcal{T})$ has a path which starts from the root and spells out $p$.

Ehrenfeucht et al. [6] introduced maximal reach pointers, which are used for efficient pattern matching queries on position heaps. Diptarama et al. [5] and Fujisato et al. [7] also introduced maximal reach pointers for their p-position heaps, and showed how efficient pattern matching queries can be done. We can naturally extend the notion of maximal reach pointers to our p-position heaps:

Definition 3* (Maximal reach pointers).*

For each $1\leq i\leq N_{p}$, the maximal reach pointer of the node $\mathit{h}_{i}$ points to the deepest node $v$ of $\mathsf{PPH}(\mathcal{T})$ such that $v$ represents a prefix of $\mathsf{spe}(w_{i})$.

The node which is pointed by the maximal reach pointer of node $\mathit{h}_{i}$ is denoted by $\mathsf{mrp}(i)$. The augmented $\mathsf{PPH}(\mathcal{T})$ is $\mathsf{PPH}(\mathcal{T})$ with the maximal reach pointers of all nodes. For simplicity, if $\mathsf{mrp}(i)$ is equal to $\mathit{h}_{i}$, then we omit this pointer. See Figure 3 for an example of augmented $\mathsf{PPH}(\mathcal{T})$.

3.3 P-matching with augmented parameterized position heap

It is straightforward that by applying Diptarama et al.’s pattern matching algorithm to our $\mathsf{PPH}(\mathcal{T})$ augmented with maximal reach pointers, parameterized pattern matching can be done in $O(m\log(\sigma+\pi)+m\pi+\mathit{pocc}^{\prime})$ time where $\mathit{pocc}^{\prime}$ is the number of nodes in $\mathsf{pCST}(\mathcal{T})$ that p-match with the pattern. Since each node in $\mathsf{pCST}(\mathcal{T})$ stores the pointers to the corresponding nodes in $\mathcal{T}$, then we can answer all the nodes that p-match with the pattern.

Diptarama et al.’s algorithm stands on Lemmas 13 and 14 of [5]. These lemmas can be extended to our $\mathsf{PPH}(\mathcal{T})$ as follows:

Lemma 2**.**

Suppose $\mathsf{spe}(p)$ is represented by a node $u$ of augmented $\mathsf{PPH}(\mathcal{T})$. Then $p$ p-matches with the prefix of length $|p|$ of $w_{i}$ iff $\mathsf{mrp}(i)$ is $u$ or a descendant of $u$.

Proof.

Let $u$ be the node in augmented $\mathsf{PPH}(\mathcal{T})$ that represents $\mathsf{spe}(p)$.

Assume that $p$ p-matches with the prefix of length $|p|$ of $w_{i}$ and the node $v$ satisfying $\mathsf{id}(v)=i$ represents $\mathsf{spe}(w_{i})[1..k]$. Then either $\mathsf{spe}(w_{i})[1..k]$ is a prefix of $\mathsf{spe}(p)$ or $\mathsf{spe}(p)$ is a prefix of $\mathsf{spe}(w_{i})[1..k]$. This implies that $v$ is either an ancestor or a descendant of $u$. If $v$ is an ancestor of $u$, then $v$ ($\mathsf{mrp}(i)$) points to $u$ or an its descendant since $\mathsf{spe}(w_{i})[1..|p|]$ is represented by $u$. By the definition of maximal reach pointers, if $v$ is a descendant of $u$, $v$ ($\mathsf{mrp}(i)$) points to $u$ or an its descendant.

Assume that $\mathsf{mrp}(i)$ is $u$ or an its descendant. Let $k$ be the integer such that $\mathsf{mrp}(i)$ represents $\mathsf{spe}(w_{i})[1..k]$. Then $\mathsf{spe}(p)$ is a prefix of $\mathsf{spe}(w_{i})[1..k]$. This implies that $p$ p-matches with the prefix of length $|p|$ of $w_{i}$. ∎

Lemma 3**.**

Suppose that $\mathsf{spe}(p)$ is not represented in augmented $\mathsf{PPH}(\mathcal{T})$. There is a factorization $q_{1},\ldots,q_{k}$ of $p$ s.t. $q_{j}$ is the longest prefix of $\mathsf{spe}(p[|q_{1}\cdots q_{j-1}|+1..|p|])$ that is represented in augmented $\mathsf{PPH}(\mathcal{T})$. If $p$ p-matches with the prefix of length $|p|$ of $w_{i}$, then $\mathsf{mrp}(i+|q_{1}\cdots q_{j-1}|)$ is the node which represents $\mathsf{spe}(q_{j})$ for any $1\leq j<k$ and $\mathsf{mrp}(i+|q_{1}\cdots q_{k-1}|)$ is the node which represents $\mathsf{spe}(q_{k})$ or a descendant of $\mathsf{mrp}(i+|q_{1}\cdots q_{k-1}|)$.

Proof.

Assume that $p=q_{1},\ldots,q_{k}$ p-matches with the prefix of length $|p|$ of $w_{i}$. Since $\mathsf{spe}(q_{1})$ is a prefix of $\mathsf{spe}(p)$, then $\mathsf{mrp}(i)$ is the node which represents $\mathsf{spe}(q_{1})$ or an its descendant. If $\mathsf{mrp}(i)$ is an descendant of the node which represents $\mathsf{spe}(q_{1})$, then $q_{1}$ is not the longest prefix of $\mathsf{spe}(p)$ that is represented in augmented $\mathsf{PPH}(\mathcal{T})$. Thus $\mathsf{mrp}(i)$ is the node which represents $\mathsf{spe}(q_{1})$. Similarly, for every $1<j<k$, $\mathsf{spe}(q_{j})$ is a prefix of $\mathsf{spe}(p[|q_{1}\cdots q_{j-1}|+1..|p|])$ and p-matches with the prefix of length $|q_{j}|$ of $w_{i}[|q_{1}\cdots q_{j-1}|+1..|w_{i}|]$. Thus $\mathsf{mrp}(i+|q_{1}\cdots q_{j-1}|)$ is the node which represents $\mathsf{spe}(q_{j})$. Finally, $\mathsf{mrp}(i+|q_{1}\cdots q_{k-1}|)$ has to be the node which represents $\mathsf{spe}(q_{k})$ or an its descendant since $q_{k}$ is a suffix of $p$. ∎

Theorem 1**.**

Using our augmented $\mathsf{PPH}(\mathcal{T})$, one can perform parameterized pattern matching queries in $O(m\log(\sigma+\pi)+m\pi+\mathit{pocc})$ time.

4 Construction of parameterized position heaps

In this section, we show how to construct the augmented $\mathsf{PPH}(\mathcal{T})$ of a given common suffix trie $\mathcal{T}$ of size $N$. For convenience, we will sometimes identify each node $v$ of $\mathsf{PPH}(\mathcal{T})$ with the string which is represented by $v$. In Section 4.1, we show how to compute $\mathsf{pCST}(\mathcal{T})$ from a given common suffix trie $\mathcal{T}$. In Section 4.2, we propose how to construct $\mathsf{PPH}(\mathcal{T})$ from $\mathsf{pCST}(\mathcal{T})$.

4.1 Computing $\mathsf{pCST}(\mathcal{T})$ from $\mathcal{T}$

Here, we show how to construct $\mathsf{pCST}(\mathcal{T})$ of a given $\mathcal{T}$ of size $N$.

Lemma 4**.**

For any common suffix trie $\mathcal{T}$ of size $N$, $\mathsf{pCST}(\mathcal{T})$ can be computed in $O(N\pi)$ time and space.

Proof.

We process every node of $\mathcal{T}$ in a breadth first manner. Let $x_{j}$ be the p-string which is represented by $j$-the node of $\mathcal{T}$. Suppose that we have processed the first $k$ nodes and have computed $\mathsf{pCST}(\mathcal{T})^{i}$ ($i\leq k$). We assume that the $j$-th node of $\mathcal{T}$, for any $1\leq j\leq k$, holds the resulting substitutions from $x_{j}$ to $\mathsf{spe}((x_{j})^{R})^{R}$ (i.e., $x_{j}[\alpha]$ is mapped to $\mathsf{spe}((x_{j})^{R})^{R}[\alpha]$), and also a pointer to the corresponding node of $\mathsf{pCST}(\mathcal{T})^{i}$ (i.e., pointer to the node representing $\mathsf{spe}((x_{j})^{R})^{R}$). We consider processing the $(k+1)$-th node of $\mathcal{T}$. Since $x_{k+1}$ is encoded from right to left, we can determine a character $\mathsf{spe}((x_{k+1})^{R})^{R}[1]$ in $O(\pi)$ time. Then, we can insert a new node that represents $\mathsf{spe}((x_{k+1})^{R})^{R}$ as a parent of the node which represents $\mathsf{spe}((x_{k+1}[2..|x_{k+1}|])^{R})^{R}$ if there does not exist such a node in $\mathsf{pCST}(\mathcal{T})^{i}$. Therefore, we can compute $\mathsf{pCST}(\mathcal{T})$ in $O(N\pi)$ time and space. ∎

4.2 Computing $\mathsf{PPH}(\mathcal{T})$ from $\mathsf{pCST}(\mathcal{T})$

For efficient construction of our $\mathsf{PPH}(\mathcal{T})$, we use reversed suffix links defined as follows.

Definition 4* (Reversed suffix links).*

For any node $v$ of $\mathsf{PPH}(\mathcal{T})$ and a character $a\in\Sigma\cup\Pi$, let

[TABLE]

See Figure 4 for an example of $\mathsf{PPH}(\mathcal{T})$ with reversed suffix links. In our algorithm, firstly, we insert a new node $h_{i}$ of $\mathsf{PPH}(\mathcal{T})^{i}$ to $\mathsf{PPH}(\mathcal{T})^{i-1}$. After that, we add new suffix links which point to $h_{i}$. When we have computed $\mathsf{PPH}(\mathcal{T})$, then we compute all maximal reach pointers of $\mathsf{PPH}(\mathcal{T})$.

4.2.1 Inserting a new node

Assume that $\mathit{c}_{j}$ (i.e., $j$-th node of $\mathsf{pCST}(\mathcal{T})$) is the child of $\mathit{c}_{i}$ for any $2\leq i\leq N_{p}$. Consider to insert $\mathit{h}_{i}$ (i.e., the node of $\mathsf{PPH}(\mathcal{T})$ which corresponds to $c_{i}$) to $\mathsf{PPH}(\mathcal{T})^{i-1}$. We show how to find the parent of $\mathit{h}_{i}$ by starting from $\mathit{h}_{j}$. There are 3 cases based on $w_{i}[1]$ as follows:

• •

$w_{i}[1]\in\Pi$ and $w_{i}[1]$ appears in $w_{j}[1..|\mathit{h}_{j}|]$ (Lemma 5),

• •

$w_{i}[1]\in\Pi$ and $w_{i}[1]$ does not appear in $w_{j}[1..|\mathit{h}_{j}|]$ (Lemma 6),

• •

$w_{i}[1]\in\Sigma$ (Lemma 7).

Lemma 5**.**

Assume that $w_{i}[1]\in\Pi$ appears in $w_{j}[1..|\mathit{h}_{j}|]$, and $a$ is the character in $\Pi$ such that $a=\mathsf{spe}(w_{j}[1..|\mathit{h}_{j}|])[\alpha]$ and $w_{i}[1]=w_{j}[\alpha]$ for some $1\leq\alpha\leq|w_{j}[1..|\mathit{h}_{j}|]|$. Let $\mathit{h}_{k}$ be the node of $\mathsf{PPH}(\mathcal{T})^{i-1}$ which is the lowest ancestor of $\mathit{h}_{j}$ that has a reversed suffix link labeled with $a$. Then, $\mathit{h}_{i}$ is a child of the node representing $\mathsf{rsl}(a,\mathit{h}_{k})$.

Proof.

Let $\ell$ be the length of $\mathsf{rsl}(a,\mathit{h}_{k})$. To prove this lemma, we show that

$\mathsf{rsl}(a,\mathit{h}_{k})=\mathsf{spe}(w_{i})[1..\ell]$, and 2. 2.

There does not exist a node which represents $\mathsf{spe}(w_{i})[1..\ell+1]$ in $\mathsf{PPH}(\mathcal{T})^{i-1}$.

By the definition of reversed suffix links and $\mathsf{spe}$, we have

[TABLE]

Thus, we have proved the first statement.

By a similar argument, we also have $\mathsf{spe}(a\cdot\mathsf{spe}(w_{j}[1..\ell]))=\mathsf{spe}(w_{i}[1..\ell+1])$. Thus, if $\mathsf{spe}(w_{i})[1..\ell+1]$ is represented in $\mathsf{PPH}(\mathcal{T})^{i-1}$, the node representing $\mathsf{spe}(w_{j}[1..\ell])$ must have a reversed suffix link labeled with $a$. This contradicts the fact that $\mathit{h}_{k}$ is the lowest ancestor of $\mathit{h}_{j}$ which has a reversed suffix link labeled with $a$. ∎

Lemma 6**.**

Assume that $w_{i}[1]\in\Pi$ does not appear in $w_{j}[1..|\mathit{h}_{j}|]$. Let $\mathit{h}_{k}$ be the node of $\mathsf{PPH}(\mathcal{T})^{i-1}$ which is the lowest ancestor of $\mathit{h}_{j}$ that has a reversed suffix link labeled with $a\in\Pi\setminus\{\mathit{h}_{j}[\alpha]\mid 1\leq\alpha\leq|\mathit{h}_{j}|\}$. Then, $\mathit{h}_{i}$ is a child of the node representing $\mathsf{rsl}(a,\mathit{h}_{k})$.

Proof.

Let $\ell$ be the length of $\mathsf{rsl}(a,\mathit{h}_{k})$. We show similar statements to the proof of the previous lemma hold, but with different assumptions on $w_{i}[1]$ and $a$. By the definition of reversed suffix links and $\mathsf{spe}$, we have

[TABLE]

Thus, we have proved the first statement.

By a similar argument, we also have $\mathsf{spe}(a\cdot\mathsf{spe}(w_{j}[1..\ell]))=\mathsf{spe}(w_{i}[1..\ell+1])$. This implies that the second statement holds (similar to the proof of the previous lemma). ∎

Lemma 7**.**

Assume that $w_{i}[1]\in\Sigma$. Let $\mathit{h}_{k}$ be the node in $\mathsf{PPH}(\mathcal{T})^{i-1}$ which is the lowest ancestor of $\mathit{h}_{j}$ that has a reversed suffix link labeled with $w_{i}[1]$. Then, $\mathit{h}_{i}$ is a child of the node representing $\mathsf{rsl}(w_{i}[1],\mathit{h}_{k})$.

Proof.

Since $w_{i}[1]\in\Sigma$, we can show the lemma in a similar way to the above proofs. ∎

4.2.2 Inserting new reversed suffix links

In our algorithm, we will add reversed suffix links which point to $h_{i}$ after inserting a new node $h_{i}$. The following lemma shows the number of nodes which point to $h_{i}$ by reversed suffix links is at most one.

Lemma 8**.**

For any node $v$ of $\mathsf{PPH}(\mathcal{T})$, the number of nodes which point to $v$ by reversed suffix links is at most one.

Proof.

Let $v_{1},v_{2}$ be nodes of $\mathsf{PPH}(\mathcal{T})$. Assume that $\mathsf{rsl}(a_{1},v_{1})=\mathsf{rsl}(a_{2},v_{2})$ for some $a_{1},a_{2}\in\Sigma\cup\Pi$ and $v_{1}\neq v_{2}$ hold. By the definition of reversed suffix links, $\mathsf{spe}(a_{1}\cdot v_{1})=\mathsf{spe}(a_{2}\cdot v_{2})$. Namely, $a_{1}\cdot v_{1}\approx a_{2}\cdot v_{2}$ holds. This implies that $v_{1}\approx v_{2}$, i.e., $\mathsf{spe}(v_{1})=\mathsf{spe}(v_{2})$. Since $v_{1}$ and $v_{2}$ are node of $\mathsf{PPH}(\mathcal{T})$, $\mathsf{spe}(v_{1})=v_{1}$ and $\mathsf{spe}(v_{2})=v_{2}$ hold. This contradicts the fact that $v_{1}\neq v_{2}$. ∎

By the above lemma and arguments of insertion, the node which points to the new node $\mathit{h}_{i}$ by reversed suffix links is only a child of $\mathit{h}_{k}$ which is an ancestor of $\mathit{h}_{j}$.

4.2.3 Construction algorithm

Finally, we explain our algorithm of constructing our position heap. From the above lemmas, we can use similar techniques to Nakashima et al. [12] which construct the position heap of a trie of normal strings. One main difference is the computation of the label of inserted edges/reversed suffix links. In so doing, each node $h_{\alpha}$ holds the resulting substitutions from $w_{\alpha}[1..|h_{\alpha}|]$ to $\mathsf{spe}(w_{\alpha}[1..|h_{\alpha}|])$. By using these substitutions, we can compute the corresponding label in $O(\pi)$ time. Thus, we can insert new nodes and new suffix links in $O(\pi)$ time for each node of $\mathsf{pCST}(\mathcal{T})$. In fact, since we need to use $(\sigma+\pi)$-copies of the position heap for nearest marked ancestor queries on each character, we use $O(\sigma+\pi)$ time to update the data structures needed for each node of $\mathsf{pCST}(\mathcal{T})$. Therefore, we have the following lemma.

Lemma 9**.**

We can compute $\mathsf{PPH}(\mathcal{T})$ from $\mathsf{pCST}(\mathcal{T})$ of size $N_{p}$ in $O(N_{p}(\sigma+\pi))$ time and space.

Therefore, we can obtain the following result by Lemmas 4 and 9.

Theorem 2**.**

We can compute $\mathsf{PPH}(\mathcal{T})$ of a given common suffix trie $\mathcal{T}$ of size $N$ in $O(N(\sigma+\pi))$ time and space.

Since we can also compute all maximal reach pointers of $\mathsf{PPH}(\mathcal{T})$ efficiently in a similar way to [12] (this algorithm is also similar to suffix link construction), we also have the following lemma.

Lemma 10**.**

We can compute all the maximal reach pointers for $\mathsf{PPH}(\mathcal{T})$ in $O(N_{p}(\sigma+\pi))$ time and space.

Hence, we can get the following result.

Theorem 3**.**

We can compute the augmented $\mathsf{PPH}(\mathcal{T})$ of a given common suffix trie $\mathcal{T}$ of size $N$ in $O(N(\sigma+\pi))$ time and space.

5 Conclusions and open problems

This paper proposed the p-position heap for a CS trie $\mathcal{T}$, denoted $\mathsf{PPH}(\mathcal{T})$, which is the first indexing structure for the p-matching problem on a trie. The key idea is to transform the input CS trie $\mathcal{T}$ into a parameterized CS trie $\mathsf{pCST}(\mathcal{T})$ where p-matching suffixes are merged. We showed that the p-matching problem on the CS trie $\mathcal{T}$ can be reduced to the p-matching problem on the parameterized CS trie $\mathsf{pCST}(\mathcal{T})$. We proposed an algorithm which constructs $\mathsf{PPH}(\mathcal{T})$ in $O(N(\sigma+\pi))$ time and working space, where $N$ is the size of the CS trie $\mathcal{T}$. We also showed that using $\mathsf{PPH}(\mathcal{P})$ one can solve the p-matching problem on the CS trie $\mathcal{T}$ in $O(m\log(\sigma+\pi)+m\pi+\mathit{pocc})$ time, where $m$ is the length of a query pattern and $\mathit{pocc}$ is the number of occurrences to report.

Examples of open problems regarding this work are the following:

• •

Would it be possible to shave the $m\pi$ term in the pattern matching time using p-position heaps? This $m\pi$ term is introduced when the depth of the corresponding path of $\mathsf{PPH}(\mathcal{T})$ is shorter the pattern length $m$ and thus the pattern needs to be partitioned into $O(\pi)$ blocks in the current pattern matching algorithm [5].

• •

Can we efficiently build the p-suffix tree for a CS trie? It is noted by Baker [1, 2] that the destination of a parameterized suffix link (p-suffix link) of the p-suffix tree can be an implicit node that lies on an edge, and hence there is no monotonicity in the chain of p-suffix links. If we follow the approach by Breslauer [3] which is based on Weiner’s algorithm [17], then we need to use the reversed p-suffix link. It is, however, unclear whether one can adopt this approach since the origin of a reversed p-suffix link may be an implicit node. Recall that in each step of construction we need to find the nearest (implicit) ancestor that has a reversed p-suffix link labeled with a given character. Since there can be $\Theta(N^{2})$ implicit nodes, we cannot afford to explicitly maintain information about the reversed p-suffix links for all implicit nodes.