Matching Patterns with Variables

TL;DR

This paper reviews recent advances in efficiently matching variable-containing patterns to words, highlighting polynomial-time solutions for specific pattern classes despite the general NP-completeness.

Contribution

It provides an overview of recent results and extensions concerning polynomial-time matching algorithms for structured variable patterns.

Findings

Efficient matching algorithms exist for certain restricted pattern classes.

Matching problem is NP-complete in general but tractable in specific cases.

Extensions of pattern matching broaden applicability to more complex patterns.

Abstract

A pattern p (i.e., a string of variables and terminals) matches a word w, if w can be obtained by uniformly replacing the variables of p by terminal words. The respective matching problem, i.e., deciding whether or not a given pattern matches a given word, is generally NP-complete, but can be solved in polynomial-time for classes of patterns with restricted structure. In this paper we overview a series of recent results related to efficient matching for patterns with variables, as well as a series of extensions of this problem.

Equations

$$((v_1,v_2,v_3),(v_2,v_4,v_5),(v_3,v_1,v_3),(v_4,v_1,v_2))$$

$$x_1\text{name:}y;\text{address:}z;x_2\text{name:}y;\text{address:}z{x}_3$$

Full text

1 Introduction

A pattern with variables, called simply pattern in the context of this work, is a string that consists of terminal symbols (e. g., $\mathtt{a},\mathtt{b},\mathtt{c}$) and variables (e. g., $x_{1},x_{2},x_{3}$). The terminal symbols are treated as constants, while the variables are to be uniformly replaced by strings over the set of terminals (i. e., different occurrences of the same variable are replaced by the same string); thus, a pattern is mapped to a terminal word. For example, $x_{1}\mathtt{a}\mathtt{b}x_{1}x_{2}\mathtt{c}x_{2}x_{1}$ can be mapped to $\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{c}\mathtt{a}\mathtt{a}\mathtt{c}\mathtt{c}\mathtt{a}\mathtt{a}\mathtt{a}\mathtt{c}$ and $\mathtt{b}\mathtt{a}\mathtt{b}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}$ by the replacements $(x_{1}\to\mathtt{a}\mathtt{c},x_{2}\to\mathtt{c}\mathtt{a}\mathtt{a})$ and $(x_{1}\to\mathtt{b},x_{2}\to\mathtt{a})$, respectively.

Patterns with variables appear in various areas of theoretical computer science, such as language theory (pattern languages [2]), learning theory (inductive inference [2, 69, 75, 24], PAC-learning [53]), combinatorics on words (word equations [51, 67], unavoidable patterns [61]), pattern matching (generalised function matching [1, 70]), database theory (extended conjunctive regular path queries [5]), and we can also find them in practice in the form of extended regular expressions with backreferences [12, 37, 31], used in programming languages like Perl, Java, Python, etc.

Generally, in all these contexts, patterns with variables are used to model various combinatorial pattern matching questions. For instance, searching for a word $w$ in a text $t$ can be expressed as testing whether the pattern $xwy$ can be mapped to $t$ and testing whether a word $w$ contains a cube is equivalent to testing whether the pattern $xy^{3}z$ can be mapped to $w$, such that $y$ is not mapped to an empty word. Not only problems of testing whether a given word contains a regularity or a motif of a certain form can be expressed by patterns, but also problems asking whether a word can be factorised in a specifically restricted manner can be modelled in this way. For instance, asking whether $x_{1}^{2}x_{2}^{2}\ldots x_{k}^{2}$ can be mapped to $w$, such that none of the variables $x_{i}$ are mapped to an empty word, is equivalent to asking whether the word $w$ can be factorised into $k$ non-empty squares.

Unfortunately, deciding whether a given arbitrary pattern can be mapped to a given word, the matching problem, is $\operatorname{\mathsf{NP}}$-complete [2], whether we ask that the variables are mapped to non-empty words or not. This intractability result severely limits the practical application of patterns. Indeed, in many tasks related to applications of patterns, the matching problem is a necessary step, so the tasks become intractable as well. For instance, this is the case for the task of computing so-called descriptive patterns for finite sets of words (see [2, 35, 36] for more information on descriptive patterns): one cannot solve this problem without solving a series of (general) pattern matching tasks [27]. A more detailed analysis of the complexity of the hardness of the matching problem will be presented in Section 3.

On the other hand, some strong restrictions on the structure of patterns yield subclasses for which the matching problem is tractable (i.e., can be solved in polynomial time). This is clearly the case of patterns where the number of different variables in the patterns is bounded by a constant, but more sophisticated and general such subclasses can be defined. We will discuss a series of results related to this topic in Sections 4.2, 5.1 and 5.2. In our analysis, the most general class of patterns which allow for a polynomial-time pattern matching problem is defined by establishing a deep connection between strings/patterns and graphs, and considering only patterns which correspond to graphs with bounded structural parameters. As such, the subclass of patterns with bounded treewidth. The question of finding classes of patterns which can be matched in polynomial time but do not have bounded treewidth seemed interesting to us. We show a natural construction of such patterns in Section 6.

We continue this survey with a result showing that considering some of the structural parameters, that lead to efficient pattern matching algorithms, as general structural parameters of strings, may lead to remarkable results in other apparently unrelated domains. We show in Section 7 how our results for strings can be used to obtain a state-of-the-art approximation algorithm for computing the cutwidth of graphs.

We conclude the survey with a series of extensions. We discuss the problem of injective pattern matching as well as the satisfiability problem for word equations with restricted form.

2 Basic Definitions

For detailed definitions regarding combinatorics on words we refer to [60].

We denote our alphabet by $\Sigma$, the empty word by $\varepsilon$, the set of all non-empty words over $\Sigma$ by $\Sigma^{+}$, the set of all words over $\Sigma$ by $\Sigma^{*}$, and the length of a word $w$ by $|w|$. $(\Sigma^{*},\cdot,\varepsilon)$ is the free monoid over $\Sigma$ with concatenation as its binary operation, written $\cdot$. For $w\in\Sigma^{*}$ and every integers $i,j$ with $1\leq i\leq j\leq|w|$, let $w[i..j]=w[i]\cdots w[j]$, where $w[k]$ represents the letter on position $k$ and $1\leq k\leq|w|$. A period of $w$ is any positive integer $p$ for which $w[i]=w[i+p]$, for all defined positions. Moreover, in this case, $w$ is said to be $p$-periodic. Its minimal period is denoted by $per(w)$ and represents the smallest period of $w$. For example, $w=\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}$ has periods $8$ and $4$; in particular, $per(w)=4$. A word $w$ is called periodic if $per(w)\leq\frac{|w|}{2}$.

The concatenation of $k$ words $w_{1},w_{2},\ldots,w_{k}$ is written $\Pi_{i=1,k}w_{i}$. If $w=w_{i}$ for all integers $i$ with $1\leq i\leq k$, this represents the $k$th power of $w$, denoted by $w^{k}$; here, $w$ is a root of $w^{k}$. We can further extend the notion of a power of a word by saying that $w=w[1..per(w)]^{\frac{|w|}{per(w)}}$. We say that $w$ is primitive if it cannot be expressed as a power of exponent $\ell$ of any root, where $\ell$ is an integer with $\ell>1$. Conversely, if $w=v^{\ell}$ for some integer $\ell>1$, then $w$ is also called a repetition. The infinite repetition $vvv\cdots$ of some word $v$ is denoted $v^{\omega}$.

For any word $w\in\Sigma^{+}$ with $w=xyz$, we say that $y$ is a factor of $w$. If $x$ is empty, then $y$ is also a prefix of $w$, while when $z$ is empty, then $y$ is also a suffix. Whenever we have a factor both as a prefix and as a suffix, the factor is said to be a border of the word. Furthermore, every word $u=yzx\in\Sigma^{+}$ is a conjugate of $w$. Note that, if $w$ is primitive, so is every conjugate of it. If $w=vu$, then $v^{-1}w=u$.

Let $X=\{x_{1},x_{2},x_{3},\ldots\}$ and call every $x\in X$ a variable. For a finite alphabet $\Sigma$ of terminals with $\Sigma\cap X=\emptyset$, we define $\operatorname{\textsf{Pat}}_{\Sigma}=(X\cup\Sigma)^{+}$ and $\operatorname{\textsf{Pat}}=\bigcup_{\Sigma}\operatorname{\textsf{Pat}}_{\Sigma}$. Every $\alpha\in\operatorname{\textsf{Pat}}$ is a pattern and every $w\in\Sigma^{*}$ is a (terminal) word. Given a word or a pattern $v$, for the smallest sets $B\subseteq\Sigma$ and $Y\subseteq X$ with $v\in(B\cup Y)^{*}$, we denote $\operatorname{\textsf{alph}}(v)=B$ and $\operatorname{\textsf{var}}(v)=Y$. For any $x\in\Sigma\cup X$ and $\alpha\in\operatorname{\textsf{Pat}}_{\Sigma}$, $|\alpha|_{x}$ denotes the number of occurrences of $x$ in $\alpha$; for the sake of convenience, we set $|\alpha|_{x}=0$ for every symbol $x$ not in $\Sigma\cup X$. For a pattern $\alpha$, we say that $w=\alpha[i..i+|w|]$ is a maximal terminal factor of $\alpha$ if $\alpha[i-1]$ and $\alpha[i+|w|+1]$ are either not defined, or are variables.

A substitution (for $\alpha$) is a mapping $h:\operatorname{\textsf{var}}(\alpha)\rightarrow\Sigma^{*}$. For every $x\in\operatorname{\textsf{var}}(\alpha)$, we say that $x$ is substituted by $h(x)$ and $h(\alpha)$ denotes the word obtained by substituting every occurrence of a variable $x$ in $\alpha$ by $h(x)$ and leaving the terminals unchanged. We say that the pattern $\alpha$ matches $w\in\Sigma^{+}$ if $h(\alpha)=w$ for some substitution $h:\operatorname{\textsf{var}}(\alpha)\rightarrow\Sigma^{*}$. Substitutions of the form $h:\operatorname{\textsf{var}}(\alpha)\rightarrow\Sigma^{+}$, i. e., the empty word is excluded from the range of the substitution, are also called non-erasing; in order to emphasize that the substitution by the empty word is allowed, we also use the term erasing substitution.

Example 1*.*

Let $\beta=x_{1}\mathtt{a}x_{2}\mathtt{b}x_{2}x_{1}x_{2}$ be a pattern and let $u=\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{b}\mathtt{a}\mathtt{b}\mathtt{b}\mathtt{b}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{b}\mathtt{b}$ and $v=\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{a}\mathtt{b}\mathtt{b}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{b}\mathtt{a}\mathtt{b}$ be terminal words. The pattern $\beta$ matches both $u$ and $v$, witnessed by the substitutions $h$ with $h(x_{1})=\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{b}$, $h(x_{2})=\mathtt{b}$ and $g$ with $g(x_{1})=g(x_{2})=\mathtt{a}\mathtt{b}$, respectively. Moreover, $\beta$ also matches the word $w=\mathtt{a}\mathtt{c}\mathtt{b}\mathtt{b}\mathtt{c}\mathtt{b}\mathtt{c}\mathtt{b}$ by the erasing substitution $h$ with $h(x_{1})=\varepsilon$, $h(x_{2})=\mathtt{c}\mathtt{b}$; it can be easily verified that there is no non-erasing substitution that maps $\beta$ to $w$.

The matching problem, denoted by $\operatorname{\textsc{Match}}$, is to decide for a given pattern $\alpha$ and word $w$, whether there exists a substitution $h$ with $h(\alpha)=w$. The variant where we are only concerned with non-erasing substitutions is called the non-erasing case of the matching problem; we also use the term erasing-case in order to emphasize that substitution by the empty word is allowed. Another special variant is the terminal-free case of the matching problem, where the input patterns are terminal-free, i. e., they do not contain any occurrences of terminal symbol. We shall briefly discuss some particularities of these different special cases of the matching problem in Section 3. Note that in the sections on efficient algorithms, namely Sections 5.1, 5.2, and 6, we only consider the non-erasing case (with terminal symbols) of the matching problem. The presented results can easily be generalised to the general setting, but we prefer the respective framework for the ease of the presentation.

For any $P\subseteq\operatorname{\textsf{Pat}}$, the matching problem for $P$ (or $\operatorname{\textsc{Match}}$ for $P$, for short) is the matching problem, where the input patterns are from $P$. In the sections of this paper we will introduce and discuss several interesting families of patterns.

As we discuss efficient algorithms, it is important to describe the computational model we use in this work. This is the standard unit-cost RAM with logarithmic word size. Also, all logarithms appearing in our time complexity evaluations are in base 2. For the sake of generality, we assume that whenever we are given as input a word $w\in\Sigma^{*}$ of length $n$, the symbols of $w$ are in fact integers from $\{1,2,\ldots,n\}$ (i.e., $\Sigma=\operatorname{\textsf{alph}}(w)\subseteq\{1,2,\ldots,n\}$), and $w$ is seen as a sequence of integers. This is a common assumption in the area of algorithmics on words (see, e.g., the discussion in [52]). Clearly, our algorithmic results hold canonically for constant alphabets, as well.

3 The Hardness of the Matching Problem

First, we recall that there are several different variants of the matching problem: the most general case (substitution by the empty word and occurrences of terminals in the patterns are possible), the non-erasing case (with terminal symbols), the terminal-free (erasing) case, and finally the terminal free non-erasing case. As we shall see, these differences do not matter too much if we are only concerned with the matching problem of patterns. However, in other contexts of patterns with variables (e. g., other decision problems, learning theory), these differences are most crucial and we therefore briefly provide some background.

For the class of the so-called pattern languages, i. e., the sets of all words that match a pattern, the difference between the erasing and the non-erasing case is important, since these classes of formal languages differ quite substantially with respect to basic decision problems. For example, in the non-erasing case, two patterns describe the same language if and only if the patterns are identical (up to a renaming of variables), while it is open whether the equivalence problem is even decidable in the erasing-case (see, e. g., Section $6$ in [68], or [74]). Moreover, the inclusion problem, which is undecidable for both the erasing and the non-erasing case (see [50, 34]), can be decided for terminal-free patterns in the erasing case, while for terminal-free non-erasing patterns the decidability status is open (intuitively speaking, this has to do with the fact that avoidability questions of the form “does pattern $\beta$ necessarily occur in long enough words over a $k$-letter alphabet?” can be expressed as inclusion for two languages given by terminal-free non-erasing patterns). Finally, also whether patterns (or descriptive patterns) can be inferred from positive data strongly depends on whether the erasing or non-erasing case is considered, or whether or not terminal symbols in the patterns are allowed (see [73, 75, 36, 35]).

For the matching problem (note that this corresponds to the membership problem for pattern languages), whether we consider erasing or non-erasing substitution, or whether or not we disallow terminal symbols in the patterns, has little impact on its computational hardness. In fact, that the matching problem for patterns with variables is $\operatorname{\mathsf{NP}}$-complete has been independently discovered in different communities and for slightly different problem variants (see, e. g., the introductions of [28, 29] for some remarks on the history of the investigation of the matching problem).

If we consider the most general case, i. e., erasing substitutions and terminals in the patterns, then a hardness-reduction is rather simple. For example, the Boolean formula

[TABLE]

in $3$-CNF (without negated variables) is $1$-in-$3$ satisfiable (i. e., satisfiable with exactly one literal per clause set to true) if and only if the following word $w$ is matched by the pattern $\alpha$:

[TABLE]

We can further observe that this simple reduction also shows that the matching problem is hard even for binary terminal alphabets and under the restriction that variables are substituted by single symbols (or the empty word) only. This directly raises the questions under which restrictions the matching problem remains hard. For example, a problem instance has a large number of natural parameters (length of the pattern, length of the word, number of variables, number of occurrences per variable, alphabet size, length of words substituted for variables) and in addition to that, it comes in four natural variants resulting from whether we consider the erasing or non-erasing case, and whether or not we allow terminals in the pattern. In the above reduction, the number of variables, the number of occurrences per each variable and the word length are unbounded.

All these numerous restricted problem variants have been thoroughly investigated in [28] and it turns out that the matching problem remains $\operatorname{\mathsf{NP}}$-hard under rather strong restrictions. We cite the following result as an example and refer to [28] for further details.

Theorem 2** ([28]).**

The erasing case of the matching problem for patterns with variables is $\operatorname{\mathsf{NP}}$-complete, even if $\Sigma=\{\mathtt{a},\mathtt{b}\}$, every variable has at most $2$ occurrences and every variable can only be substituted by a single symbol or the empty word.

This result also holds as stated for terminal-free patterns. In the non-erasing case, however, it holds when the bound on the substitution words is $3$ instead of $1$, and in the non-erasing and terminal-free case the result holds when additionally the bounds on the occurrences per variable and alphabet size are $3$ and $4$, respectively.

The only polynomial-time solvable cases of the matching problem obtained by restricting the numerical parameters mentioned above are trivial ones. More precisely, the matching problem can be easily solved for unary alphabets (in this case, we only have to solve an equation in the integers and with integer coefficients, which are given in unary encoding), or if every variable has only one occurrence (the patterns are then regular, see Section 4), or if the number of variables or the length of the input word is bounded by a constant (the former is obvious, while the latter, in the erasing case, requires a slightly more careful argument [43]).

In particular, this also points out that Theorem 2 describes some kind of dichotomy, i. e., if we would further restrict the alphabet size, or the maximum number of occurrences per variable to $1$, then we would obtain a polynomial-time solvable variant (even if all other parameters are unrestricted); similarly, if we allow variables to be substituted by single symbols only, but not the empty word, then the matching problem becomes efficiently solvable as well (regardless of the alphabet size).

Generally, by brute-force algorithms, the matching problem can be solved in time $|\alpha|^{\operatorname{O}(|w|)}$ or $|w|^{\operatorname{O}(|\alpha|)}$, making it polynomial-time solvable provided that there is a constant upper bound on $|w|$ or $|\alpha|$ (in fact, a bound on $|\operatorname{\textsf{var}}(\alpha)|$ is sufficient). However, this constant upper bound occurs in the exponent, which means that even for rather low such bounds, say $7$, the corresponding polynomial-time algorithms are most likely impractical for larger problem instances. This leads to the question whether exponential-time algorithms are possible whose running-times are such that the exponential part exclusively depends on, say $|\operatorname{\textsf{var}}(\alpha)|$, but not on $|w|$, i. e., running-times of the form $f(|\operatorname{\textsf{var}}(\alpha)|)\times g(|\alpha|,|w|)$, where $g$ is a polynomial and $f$ is some computable function (exponential, or even double-exponential etc.). Such a running-time is polynomial for upper bounded $|\operatorname{\textsf{var}}(\alpha)|$, but the degree of the polynomial is always the same independent from the actual upper bound. If a problem has an algorithm with such a running-time, then it is called fixed-parameter tractable (with respect to the bounded parameter); see the textbooks [23, 30] for more information on parameterised complexity. Whether the matching problem for patterns with variables allows fixed-parameter tractability for some parameters has been thoroughly investigated in [29]. Although there are some more or less trivial cases of fixed-parameter tractability, the main insight provided by [29] is of a negative nature and can be summarised in the following way.

Theorem 3** ([29]).**

All variants of the matching problem parameterised by $|\alpha|$ are $\operatorname{\mathsf{W}}[1]$-hard. The erasing case of the matching problem parameterised by $|w|$ is $\operatorname{\mathsf{W}}[1]$-hard.111Problems that are hard for the parameterised complexity class $\operatorname{\mathsf{W}}[1]$ are strongly believed to be not fixed-parameter tractable.

Note that since $|\alpha|$ and $|w|$ are rather general parameters, this result covers other parameters as well, e. g., $|\Sigma|$ or $|\operatorname{\textsf{var}}(\alpha)|$. In the non-erasing case, $|w|$ is an upper bound for $|\operatorname{\textsf{var}}(\alpha)|$; thus, treating $|w|$ as a parameter means that $|\operatorname{\textsf{var}}(\alpha)|$ is also a parameter and therefore the matching problem is fixed-parameter tractable by the obvious brute-force algorithm. We refer to [29] for further such simple fixed-parameter tractable case.

Consequently, even strong restrictions of the obvious numerical parameters of instances of the matching problem, i. e., number of variables, alphabet size, occurrences per variable etc., does not yield interesting efficiently matchable subclasses of patterns with variables. However, as discussed in the next section, looking deeper into the structure of patterns will help.

4 Structural Restrictions for Patterns

From an intuitive point of view it is clear that not only the mere length of a pattern or the number of its variables should have an impact on the matching complexity, but also the actual order of the variables. For example, it has been observed rather early in [79] that if the variable occurrences in the patterns are sorted, e. g., as in $x_{1}\mathtt{a}x_{1}x_{2}x_{2}\mathtt{a}\mathtt{b}x_{2}x_{2}\mathtt{a}x_{3}x_{4}\mathtt{c}x_{4}$, then they can be matched efficiently “from-left-to-right” (more precisely, it is observed in [79] that matching such patterns can be done in logarithmic space).

A systematic investigation of such structural restrictions has been done in the last decade and numerous efficiently matchable subclasses of patterns have been found. In the following, we first present a unifying approach based on graph morphisms and the concept of treewidth. Then, we define and summarise several structural parameters for patterns and respective subclasses of patterns.

4.1 Pattern Matching by Graph Morphisms

The following general framework for matching patterns with variables has been developed in [76]. For a pattern $\alpha\in(X\cup\Sigma^{*})$, the standard graph representation of $\alpha$ is the undirected graph $\mathcal{G}^{\operatorname{\mathsf{pat}}}_{\alpha}=(V_{\alpha},E_{\alpha})$, where $V_{\alpha}=\{1,2,\ldots,|\alpha|\}$ and $E_{\alpha}=E^{\operatorname{\mathsf{equ}}}_{\alpha}\cup E^{\operatorname{\mathsf{nei}}}_{\alpha}$ with $E^{\operatorname{\mathsf{equ}}}_{\alpha}=\{\{i,i+1\}\mid 1\leq i\leq|\alpha-1|\}$ being the set of neighbour edges and $E^{\operatorname{\mathsf{equ}}}_{\alpha}=\{\{i,j\}\mid\alpha[i..j]=x\beta x,x\in X,|\beta|_{x}=0\}$ being the set of equality edges (see Figure 1 for an illustration).

In a similar way, we can also encode words $w\in\Sigma^{*}$ as graph structures $\mathcal{G}^{\operatorname{\mathsf{wo}}}_{w}$, where every factor $w[i..j]$ of $w$ is represented by a vertex $(i,j)$, equality edges are drawn between $(i,j)$ and $(i^{\prime},j^{\prime})$ if $w[i..j]=w[i^{\prime}..j^{\prime}]$, and neighbour edges if $j+1=i^{\prime}$. It has been shown in [76] that $\alpha$ matches $w$ if and only if there is a graph morphism from $\mathcal{G}^{\operatorname{\mathsf{pat}}}_{\alpha}$ to $\mathcal{G}^{\operatorname{\mathsf{wo}}}_{w}$. Moreover, the concept of the treewidth for graphs now also applies to patterns (i. e., the treewidth of a pattern is the treewidth of its standard graph representation), which is of relevance since the graph morphism problem can be solved in polynomial-time provided that the source graphs have bounded treewidth.222See [23, 30] for a formal definition of the treewidth. Consequently, we can conclude the following algorithmic meta-theorem.

Theorem 4** ([76]).**

If a class $P$ of patterns has bounded treewidth, then the matching problem for $P$ can be solved in polynomial-time.

Due to the generality of the statement of Theorem 4, the polynomial-time matching algorithm that it implies is of little practical value, even for rather simple classes of patterns. On the other hand, its theoretical relevance is demonstrated by the fact that it covers almost all known classes of patterns with a polynomial-time matching problem.333See Section 6 for the respective exceptions. After an additional remark regarding [76], we shall briefly define and compare those efficiently matchable classes of patterns in the next subsection.

Remark 5*.*

Technically, the matching problem reduces to the morphism problem for (simple) relational structures instead of undirected graphs. However, since we are here only interested in the treewidth of these structures, we can as well only talk about the underlying undirected graphs.

Moreover, the actual meta-theorem of [76] is stronger in the sense that there the treewidth of patterns is not defined with respect to the standard graph representation, but with respect to a slightly more general graph representations (i. e., we allow any way of drawing the equality edges as long as all vertices corresponding to the same variable form a connected component).

4.2 Efficiently Matchable Classes of Patterns

The most obvious way to restrict patterns is to limit their number of (repeated) variables or the number of occurrences per variable. In this regard, let $\operatorname{\textsf{var}}_{k}$ and $\operatorname{\textsf{repv}}_{k}$ be the class of patterns with at most $k$ variables and with at most $k$ repeated variables, respectively. Due to Theorem 2, we already know that bounding the number of occurrences per variable does not in general yield polynomial-time matchable classes. The only exception are patterns with at most one occurrence per variable, which are called regular patterns and are denoted by $\operatorname{\textsf{reg}}$, e. g., $x_{1}\mathtt{a}x_{2}\mathtt{b}\mathtt{a}\mathtt{c}x_{3}\mathtt{a}$ is a regular pattern. Regular patterns have been first considered in [79] and their name is motivated by the fact that the corresponding pattern languages are regular languages.

Next, we define the so-called scope coincidence degree (see [76]). For every $y\in\operatorname{\textsf{var}}(\alpha)$, the scope of $y$ in $\alpha$ is defined by $\operatorname{sc}_{\alpha}(y)=\{i,i+1,\ldots,j\}$, where $i$ is the leftmost and $j$ the rightmost occurrence of $y$ in $\alpha$. The scopes of some variables $y_{1},y_{2},\ldots,y_{k}\in\operatorname{\textsf{var}}(\alpha)$ coincide in $\alpha$ if $\bigcap_{1\leq i\leq k}\operatorname{sc}_{\alpha}(y_{i})\neq\emptyset$. By $\operatorname{\textsf{scd}}(\alpha)$, we denote the scope coincidence degree of $\alpha$, which is the maximum number of variables in $\alpha$ such that their scopes coincide, and by $\operatorname{\textsf{scd}}_{k}$, we denote the class of patterns with scope coincidence degree of at most $k$. See Figure 2 for an example of the scope coincidence degree. An important special class is $\operatorname{\textsf{scd}}_{1}$, which has been first introduced in [79] as the class of non-cross patterns (denoted by $\operatorname{\textsf{nc}}$). Intuitively speaking, the variables in non-cross patterns are sorted, e. g., $x_{1}\mathtt{a}x_{1}x_{2}\mathtt{b}\mathtt{a}x_{2}\mathtt{c}x_{3}\mathtt{a}x_{3}x_{3}$.

Next, we define the locality number, which is a general string-parameter, and which has been first introduced in [15]. A word is $k$-local if there exists an order of its symbols such that, if we mark the symbols in the respective order (which is called a marking sequence), at each stage there are at most $k$ contiguous blocks of marked symbols in the word. This $k$ is called the marking number of that marking sequence. The locality number of a word is the smallest $k$ for which that word is $k$-local, or, in other words, the minimum marking number over all marking sequences. For example, the marking sequence $\sigma=(\mathtt{a},\mathtt{g},\mathtt{c})$ marks $w=\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}\mathtt{c}\mathtt{a}\mathtt{c}$ as follows (marked blocks are illustrated by overlines): $\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}\mathtt{c}\mathtt{a}\mathtt{c}$, $\overline{\mathtt{a}}\mathtt{g}\overline{\mathtt{a}}\mathtt{g}\mathtt{c}\overline{\mathtt{a}}\mathtt{c}$, $\overline{\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}}\mathtt{c}\overline{\mathtt{a}}\mathtt{c}$, $\overline{\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}\mathtt{c}\mathtt{a}\mathtt{c}}$; thus, the marking number of $\sigma$ is $3$. In fact, all marking sequences for $w$ have a marking number of $3$, except $(\mathtt{g},\mathtt{a},\mathtt{c})$, for which it is $2$: $\mathtt{a}\overline{\mathtt{g}}\mathtt{a}\overline{\mathtt{g}}\mathtt{c}\mathtt{a}\mathtt{c}$, $\overline{\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}}\mathtt{c}\overline{\mathtt{a}}\mathtt{c}$, $\overline{\mathtt{a}\mathtt{g}\mathtt{a}\mathtt{g}\mathtt{c}\mathtt{a}\mathtt{c}}$. Thus, the locality number of $w$, denoted by $\operatorname{\textsf{loc}}(w)$, is $2$. When we measure the locality number for patterns, we simply ignore all terminal symbols, e. g., $\operatorname{\textsf{loc}}(\mathtt{a}\mathtt{b}x_{1}x_{2}\mathtt{a}x_{1}x_{2}\mathtt{c}x_{3}x_{1}\mathtt{a}x_{3})=\operatorname{\textsf{loc}}(x_{1}x_{2}x_{1}x_{2}x_{3}x_{1}x_{3})=2$. The class of patterns with locality number at most $k$ is denoted by $\operatorname{\textsf{loc}}_{k}$.

The next classes have been first considered in [76] and are based on possible nesting structures of variables. For a pattern $\alpha$, we call two variables $x,y\in\operatorname{\textsf{var}}(\alpha)$ entwined if $\alpha$ contains $xyxy$ or $yxyx$ as a subsequence. A pattern $\alpha$ is nested, if no two variables in $\alpha$ are entwined; the class of nested patterns is denoted by $\operatorname{\textsf{nest}}$. A proper subclass of $\operatorname{\textsf{nest}}$, considered in [15], are the so-called strongly nested patterns (denoted by $\operatorname{\textsf{snest}}$), which are inductively defined as follows: any pattern $\alpha\in\operatorname{\textsf{var}}_{1}$ is strongly nested; if $\alpha_{1}$ and $\alpha_{2}$ are strongly nested and variable-disjoint patterns, $x$ is a variable not in $\operatorname{\textsf{var}}(\alpha_{1})\cup\operatorname{\textsf{var}}(\alpha_{2})$ and $\beta_{1},\beta_{2}\in(\{x\}\cup\Sigma)^{*}$, then $\alpha_{1}\alpha_{2}$ and $\beta_{1}\alpha_{1}\beta_{2}$ are strongly nested patterns. For example, the pattern $\alpha=x_{1}x_{2}\mathtt{a}x_{2}x_{1}\mathtt{b}x_{3}x_{4}\mathtt{a}x_{3}$ is strongly nested, whereas $\alpha x_{1}$ is nested, but not strongly nested anymore.

If, for every $x,y\in\operatorname{\textsf{var}}(\alpha)$, $\alpha=\beta x\gamma_{1}y\gamma_{2}x\gamma_{3}y\delta$ implies $\gamma_{2}=\varepsilon$, then $\alpha$ is called closely entwined, and a pattern $\alpha$ is mildly entwined if it is closely entwined and, for every $x\in\operatorname{\textsf{var}}(\alpha)$, if $\alpha=\beta x\gamma x\delta$ with $|\gamma|_{x}=0$, then $\gamma$ is nested. We denote the class of mildly entwined patterns by $\operatorname{\textsf{ment}}$. The main motivation for the somewhat peculiar class of mildly entwined patterns is that mildly entwined patterns are exactly those patterns that have a standard graph representation that is outer-planar (see [76]).444A graph is outer-planar if it has a planar embedding with all vertices lying on the outer face. It is known that outer-planar graphs have a rather low treewidth of at most $2$. Since the concept of outer-planarity generalises to $k$-outer-planarity and $k$-outer-planar graphs have a treewidth of at most $3k-1$, we can also define the classes $\operatorname{\textsf{outp}}_{k}$ of $k$-outer-planar patterns (i. e., their standard graph representation is $k$-outer-planar). In this regard note that $\operatorname{\textsf{outp}}_{1}=\operatorname{\textsf{ment}}$. See Figure 3 for an example of a mildly-entwined pattern.

It can be easily verified that all of the pattern classes defined above have bounded treewidth; thus, by application of Theorem 4, they can be matched efficiently. For some of them this upper bound on the treewidth is rather low (e. g., $\operatorname{\textsf{reg}}$, $\operatorname{\textsf{nc}}$, $\operatorname{\textsf{ment}}$), while for those classes obtained by bounding a structural parameter, e. g., $\operatorname{\textsf{repv}}_{k}$, $\operatorname{\textsf{scd}}_{k}$, $\operatorname{\textsf{loc}}_{k}$, the bound on the treewidth also grows with this parameter. Figure 4 shows how these pattern classes relate to each other and how they form infinite hierarchies within the class of all patterns (denoted by $\operatorname{\textsf{Pat}}$).

In a sense, Figure 4 is a “tractability map” for the matching problem of patterns with variables. For the classes that have low treewidth, we can expect matching algorithms that are rather efficient. On the other hand, these classes are quite restricted (compared to the full class of patterns) and are most likely only applicable for very special pattern matching tasks. An obvious approach to matching general patterns would be to first perform a preprocessing that identifies a “low class” of the tractability map that contains the input pattern and then uses the most efficient algorithm for matching it. In this regard, it is even an asset that most of the different efficiently matchable classes and hierarchies of classes are incomparable: it is possible that an input pattern has a very large locality number of $100$, but can nevertheless be matched efficiently, because its standard graph representation is $2$-outerplanar; on the other hand, a pattern could have a large scope coincidence degree and a large number of variables, but at the same time a very low locality number. It might even be a worthwhile research task to experimentally analyse a large corpus of (random) patterns with respect to the classes of the tractability map in which they are contained.

Remark 6*.*

Bounding the structural parameters defined above yields polynomial-time matchable classes of patterns; thus, the question arises whether the matching problem is also fixed-parameter tractable with respect to those parameters. However, Theorem 3 already states that this is most likely not the case for parameter $|\operatorname{\textsf{var}}(\alpha)|$, and since $|\operatorname{\textsf{var}}(\alpha)|$ is an upper bound for the number of repeated variables, the scope coincidence degree, the outer-planarity and the locality number of $\alpha$, it is also highly unlikely that we can achieve fixed-parameter tractability with respect to those parameters.

4.3 Computing Structural Parameters for Patterns

Since the structural restrictions of patterns surveyed above are all meant to be exploited algorithmically, the task of checking them (or computing the respective parameters) is an important issue. In this regard, note that in general computing the treewidth of a graph is an $\operatorname{\mathsf{NP}}$-hard problem and it is also not known whether it can be computed efficiently for standard graph representations of patterns. This also emphasises the importance of easily computable parameters that are bounding the treewidth of a pattern and also points out why the value of Theorem 4 is of a theoretical nature that provides guidance in finding such restrictions with higher practical relevance. Restrictions like the regularity, the non-cross condition, number of (repeated) variables and the different nesting properties can be easily checked for. Moreover, also the scopes of a pattern and therefore its scope coincidence degree can be efficiently computed, and the smallest $k$ for which a graph is $k$-outerplanar can also be computed in polynomial time (for more details see [77]). On the other hand, computing the locality number seems more difficult and it was left open in [15] whether or not is is hard to compute. This gap was closed in [13] where it was shown that computing the locality number is $\operatorname{\mathsf{NP}}$-hard, but fixed-parameter tractable (if the locality number or $|\Sigma|$ is considered a parameter); in addition, approximation of the locality number has also been investigated in [13] (note that these result will be discussed in more detail in Section 7).

5 Faster Pattern Matching

In this section we will overview some efficient matching algorithms developed for various classes of patterns, some defined already in the previous sections, and some defined via some other natural structural restrictions. Most of the result of this paper were shown in [26, 15, 16].

5.1 Patterns with Low Scope Coincidence Degree

We start with several definitions. The one-variable blocks in a pattern are maximal contiguous blocks of occurrences of the same variable. A pattern $\alpha$ with $m$ one-variable blocks can be written as $\alpha=w_{0}\Pi_{i=1,m}(z_{i}^{k_{i}}w_{i})$ with $z_{i}\in\operatorname{\textsf{var}}(\alpha)$ for $i\in\{1,2,\ldots,m\}$ and $z_{i}\neq z_{i+1}$, whenever $w_{i}=\varepsilon$ for $i\in\{1,2,\ldots,m-1\}$. The number of one-variable blocks is a natural complexity measure that we will consider.

Example 7*.*

The pattern $\alpha=x_{1}x_{2}x_{2}\mathtt{a}x_{2}x_{2}x_{2}x_{3}\mathtt{a}x_{3}x_{2}x_{2}x_{3}x_{3}$ has the following $7$ one-variable blocks: $x_{1},x_{2}x_{2},x_{2}x_{2}x_{2},x_{3},x_{3},x_{2}x_{2},x_{3}x_{3}$.

As discussed in the previous sections, prominent subclasses of patterns for which $\operatorname{\textsc{Match}}$ can be solved in polynomial time are the classes of patterns with a bounded number of (repeated) variables ($\operatorname{\textsf{var}}_{k}$ and $\operatorname{\textsf{repv}}_{k}$), of regular patterns ($\operatorname{\textsf{reg}}$), of non-cross patterns ($\operatorname{\textsf{nc}}$), and of patterns with a bounded scope coincidence degree ($\operatorname{\textsf{scd}}_{k}$). However, the known respective algorithms are rather poor considering their running times. For example, for $\operatorname{\textsf{var}}_{k}$, the matching problem can be solved in $\operatorname{O}(\frac{mn^{k-1}}{(k-1)!})$, where $m$ and $n$ are the lengths of the pattern and the word (see [45]). For patterns with a scope coincidence degree of at most $k$, an $\operatorname{O}(mn^{2(k+3)}(k+2)^{2})$ time algorithm can be derived using the general matching technique described by Theorem 4, where $m$ and $n$ are the lengths of the pattern and the word, respectively, and the proof that the matching problem for non-cross patterns is in $\operatorname{\mathsf{P}}$ (see [79]) leads to an $\operatorname{O}(n^{4})$-time algorithm. Hence, for all these classes, we consider the following refinement of the problem of showing that the matching problem for a class of patterns is in $\operatorname{\mathsf{P}}$.

Problem 8*.*

Let $K$ be a class of patterns for which the matching problem can be solved in polynomial time. Find an efficient algorithm that solves the matching problem for $K$.

The main class of patters considered in the following is that of patterns with bounded scope coincidence degree, and its subclasses.

If the scope coincidence degree is bounded by $1$, i. e., non-cross patterns, we can decide whether a pattern $\alpha$ having $m$ one-variable blocks matches a word $w$ of length $n$ in $\operatorname{O}(mn\log n)$ time. This result can be achieved via a general dynamic programming approach, which tries to match prefixes of the pattern $\alpha$ to the prefixes of the word $w$. This general approach is rather standard but the big gain is that it can be implemented efficiently by a a detailed combinatorial analysis of the possible matches between the one-variable blocks occurring in $\alpha$ and factors of $w$. For instance, if the shortest factor of $\alpha$ containing all occurrences of a variable $x$ starts with a one-variable block containing at least two occurrences the variable $x$, we can efficiently find the matches of this factor by exploiting a major result from [14], which states that the primitively rooted squares contained in a word of length $n$ can be listed optimally in $\operatorname{O}(n\log n)$. As each match for a factor starting with two occurrences of a variable starts with a primitively rooted square, the respective matches can be found efficiently. The result regarding primitively rooted squares can be extended to show that, given a word $w$ of length $n$ and a word $v$ with length shorter than $n$, the word $w$ contains $\operatorname{O}(n\log n)$ factors of the form $uvu$ with $uv$ primitive, and all these factors can be found optimally in $\operatorname{O}(n\log n)$ time. This allows us to find efficiently the matches for one-variable that the shortest factor of $\alpha$ which contains all occurrences of $x$ and starts with $xvx$, for all choices of a variable $x$ such that $v$ is a non-empty terminal string.

Theorem 9** ([26]).**

The matching problem for $\operatorname{\textsf{nc}}$ is solvable in $\operatorname{O}(mn\log n)$ time, where $w$ is the input word of length $n$ and $m$ is the number of one-variable blocks occurring in the pattern.

Two particular subclasses of non-cross patterns are of interest: the regular patterns $\operatorname{\textsf{reg}}$ and the one-variable patterns $\operatorname{\textsf{var}}_{1}$ (see also Figure 4). It is not hard to show that regular patterns can be matched in linear time $\operatorname{O}(|\alpha|+|w|)$, by iteratively using the Knuth-Morris-Pratt algorithm to identify greedily the terminal factors occurring in the pattern, in their orders of occurrences. All factors of a word $w$ that match a given regular pattern $\alpha$ can be detected in linear time too.

More interesting is the case of one-variable patterns. The simplest example of one-variable patterns are the repetitions, i.e., patterns of the form $x^{k}$. Checking whether a word is a match for a pattern $x^{k}$ can be done in linear time. Moreover, a compact representation of all periodic factors of a word $w$ can be also obtained in linear time by identifying the (at most $|w|$) so-called runs inside $w$ [4]. With this, a compact representation of occurrences of $x^{k}$ in $w$ can also be obtained in linear time. More complex one-variable patterns are the pseudo-repetitions (see [42, 39, 41] and the references therein). These are patterns from $\{x,x^{R}\}^{*}$, where $x^{R}$ is a variable that is always substituted by the reverse image of the string substituting $x$. Checking whether a string matches a given pseudo-repetition can be done in linear time [42]. The following general result can be shown for one-variable patterns, see [57]. Given a pattern $\alpha=v_{1}xv_{2}x\cdots v_{r-1}xv_{r}$ such that $x$ is a variable and $v_{1},v_{2},\ldots,v_{r}$ are terminal strings, a compact representation of all $P$ instances of $\alpha$ in the input string $w$ of length $n$ can be computed in $\operatorname{O}(rn)$ time, so that one can report those occurrences in $\operatorname{O}(P)$ time. The same result holds also for the case when some of the occurrences of $x$ in such a pattern are replaced by $x^{R}$. It is worth noting that using this algorithm to find the factors of a given word that match the shortest factor of $\alpha$ containing all occurrences of a variable $x$ inside a non-cross pattern in our approach for matching $\operatorname{\textsf{nc}}$ does not lead to a faster matching algorithm in that case.

When considering general patterns with bounded scope coincidence degree, one can show, using a similar dynamic programming approach as in the case of non-cross patterns, that the matching problem for $\operatorname{\textsf{scd}}_{k}$ is solvable in $\operatorname{O}(\frac{mn^{2k}}{((k-1)!)^{2}})$ time, where $n$ is the length of the input word and $m$ is, again, the number of one-variable blocks occurring in the pattern. One should note that in this case it seems hard to use the combinatorial insights used for non-cross patterns (thus, the $\log n$ factor is replaced by an $n$ factor in the evaluation of the time complexity), but, still, this algorithm is significantly faster than the previously known solution.

Theorem 10** ([26]).**

The matching problem for $\operatorname{\textsf{scd}}_{k}$ is solvable in $\operatorname{O}\left(\frac{mn^{2k}}{((k-1)!)^{2}}\right)$ time, where $w$ is the input word of length $n$ and $m$ is the number of one-variable blocks occurring in the pattern.

Next we consider the classes $\operatorname{\textsf{repv}}_{k}$. For the basic case of $k=1$, the matching problem can be solved in $\operatorname{O}(n^{2})$ time, where $n$ is the length of the input word. The idea of this algorithm is to guess the length $\ell$ of the repeated variable $x$, and then to partition the suffix array of the input word into clusters, such that all suffixes in a cluster start with the same factor of length $\ell$. Essentially, in a match between the pattern and the word, where $x$ is mapped to a factor of length $\ell$, the positions where the factors matching $x$ occur in the input word belong to the same cluster. Using this idea, the desired complexity is then reached, again via dynamic programming.

Theorem 11** ([26]).**

*The matching problem for $\operatorname{\textsf{repv}}_{k}$ is solvable in quadratic time. *

Further, one can use this result to show that the matching problem for the general class of patterns $\operatorname{\textsf{repv}}_{k}$ is solvable in $\operatorname{O}(\frac{n^{2k}}{((k-1)!)^{2}})$ time. This algorithm is better than the one that could have been obtained by using the fact that patterns with at most $k$ repeated variables have the scope coincidence degree bounded by $k+1$, and then directly applying our previous algorithm solving the matching problem for $\operatorname{\textsf{scd}}_{k+1}$.

Theorem 12** ([26]).**

The matching problem for $\operatorname{\textsf{repv}}_{k}$ is solvable in $\operatorname{O}\left(\frac{n^{2k}}{((k-1)!)^{2}}\right)$ time, where $n$ is the length of the input word.

Note that the classes of non-cross patterns and of patterns with a bounded scope coincidence degree or with a bounded number of repeated variables are of special interest, since for them we can compute so-called descriptive patterns (see [2, 79]) in polynomial time. A pattern $\alpha$ is descriptive (with respect to, say, non-cross patterns) for a finite set $S$ of words if it can generate all words in $S$ and there exists no other non-cross pattern that describes the elements of $S$ in a better way. Computing a descriptive pattern, which is $\operatorname{\mathsf{NP}}$-complete in general, means to infer a pattern common to a finite set of words, with applications for inductive inference of pattern languages (see [69]). For example, our algorithm for computing non-cross patterns can be used in order to obtain an algorithm that computes a descriptive non-cross pattern in time $\operatorname{O}(\sum_{w\in S}(m^{2}|w|\log|w|))$, where $m$ is the length of a shortest word of $S$ (see [27] for details).

The algorithms, except the ones for the basic cases of regular and non-cross patterns and patterns with only one repeated variable, still have an exponential dependency on the number of repeated variables or the scope coincidence degree. Therefore, only for very low constant bounds on these parameters can these algorithms be considered efficient. Naturally, finding a polynomial time algorithm for which the degree of the polynomial does not depend on the number of repeated variables or on the scope coincidence degree would be desirable. However, by Remark 6 such algorithms are very unlikely.

Finally we recall a result regarding gapped repeats and palindromes. A gapped repeat (palindrome) is an instance of a terminal-free pattern $xyx$ (respectively, $xyx^{R}$). For $\alpha\geq 1$, an $\alpha$-gapped repeat in a word $w$ is a factor $uvu$ of $w$ such that $|uv|\leq\alpha|u|$; the two factors $u$ in such a repeat are called arms, while the factor $v$ is called gap. Such a repeat is called maximal if its arms cannot be extended simultaneously with the same symbol to the right or, respectively, to the left. In a sense, $\alpha$-gapped repeats are instances of the pattern $xyx$ where length constraints are imposed on the strings that substitute $x$ and $y$. In [40] it was shown that the number of maximal $\alpha$-gapped repeats that may occur in a word is upper bounded by $18\alpha n$. Using this, an algorithm finding all the maximal $\alpha$-gapped repeats of a word in $\operatorname{O}(\alpha n)$ was defined; this result is optimal, in the worst case, as there are words that have $\Theta(\alpha n)$ maximal $\alpha$-gapped repeats. Comparable results were developed for the case of $\alpha$-gapped palindromes, i.e., factors $uvu^{R}$ with $|uv|\leq\alpha|u|$. On the one hand, these results were relevant as they provided optimal algorithms for the identification of $\alpha$-gapped repeats and palindromes, and closed an open problem from [55, 56] (see also [40] and the references therein for more on gapped repeats and palindromes). On the other hand, they point towards the study of $\operatorname{\textsc{Match}}$ for patterns with (linear) length constraints on the images of the variables.

5.2 Patterns with Low Locality Number

Intuitively, the notion of $k$-locality (already introduced in Section 4.2) involves marking the variables in the pattern in some arbitrary order until all the variables are marked. The pattern is $k$-local if this marking can be done while never creating more than $k$ marked blocks. Variables which only occur adjacent to those which are already marked can be marked “for free” – without creating any new blocks, and thus a valid marking sequence allows a sort-of parsing of the pattern whilst maintaining a degree of closeness (locality) to the parts already parsed. The notion of $k$-locality was introduced and further analysed in [15]. With respect to pattern matching, the main result proven in that paper is the following:

Theorem 13** ([15]).**

$\operatorname{\textsc{Match}}$* for $\operatorname{\textsf{loc}}_{k}$ can be decided in $O(mkn^{\max{(3k-1,2k+1)}})$ time, where $m$ is the length of the input pattern and $n$ is the length of the input word.*

To solve the matching problem for $\operatorname{\textsf{loc}}_{k}$ we use the following idea. Using a simple dynamic programming approach we can show that, given a pattern $\beta\in(X\cup\Sigma)^{*}$ of length $m$, we can decide in $O(m^{2k}k)$ time whether $\beta\in\operatorname{\textsf{loc}}_{k}$, and if the answer is positive, we can produce in the same time a marking sequence witnessing that $\beta$ is $k$-local. As such we can keep track of the marked factors in the pattern, while executing the marking according to the computed marking sequence. We also need now to keep track to which factors of the input word the marked factors correspond. Then we try to assign every new variable so that it fits nicely around the already matched factors. This is done efficiently using a data structure from [57], mentioned also above: given a word $w$ and a one-variable pattern $\gamma$ (so, $|\operatorname{\textsf{var}}(\gamma)|=1$), one can produce a compact representation of all the $g$ factors of $w$ matching $\gamma$ in $O(|\gamma||w|)$ time; moreover, we can obtain all the $g$ factors of $w$ matching $\gamma$ in $O(|g|)$ time. This allows us to test efficiently which factors of $w$ match any of the one-variable blocks of $\beta$, and, ultimately, to assign a value to each variable. In comparison to the algorithm from [76] for patterns of bounded treewidth, which firstly constructs relational structures from $\alpha$ and $w$, and solves the homomorphism problem on these relational structures (see Section 4.1), the above algorithm exploits directly the locality structure present in the patterns. The advantage of this more focussed approach is that it allows for a considerable improvement in the required time, reducing the exponent of $n$ from $4k+4$ to $3k-1$.

6 Efficient Pattern Matching Beyond Bounded Treewidth

In [16] the authors tried to identify classes of patterns that do not have bounded treewidth but can still be matched in polynomial time. The idea behind defining such classes was relatively simple: consider generalised repetitions of patterns.

One simple observation is that, if we can match patterns from a class $\mathcal{C}$ in polynomial time, then we can also match repetitions of these patterns in polynomial time: if we wish to check whether $\alpha^{k}$ matches a word $w$, where $\alpha$ is chosen from the class $\mathcal{C}$ for which we can solve $\operatorname{\textsc{Match}}$ efficiently, then we can firstly check whether $w=v^{k}$ for some word $v$, and then check whether $\alpha$ matches $v$, so we can also match $\alpha^{k}$ efficiently. Moreover, it can be observed that most parameters that lead to efficiently matchable classes, e. g., the scope coincidence degree or locality, are defined independently from the terminal symbols, i. e., via the word obtained after removing all terminals, which shall be called skeleton in the following (e. g., the skeleton of $x_{1}\mathtt{a}x_{2}\mathtt{b}\mathtt{a}x_{1}x_{2}\mathtt{b}$ is $x_{1}x_{2}x_{1}x_{2}$). As a result, it is possible that a pattern, that is not a repetition of any $\alpha\in\mathcal{C}$, has nevertheless a skeleton that is a repetition of a skeleton from $\mathcal{C}$. For example, $\mathtt{a}x_{1}(x_{2})^{3}x_{3}\mathtt{b}x_{3}x_{1}(x_{2})^{2}\mathtt{b}x_{2}\mathtt{a}(x_{3})^{2}$ is not a repetition of a non-cross pattern, but its skeleton $(x_{1}(x_{2})^{3}(x_{3})^{2})^{2}$ is. In [16] it is shown that, for some important classes ${\mathcal{C}}$ of patterns, including $\operatorname{\textsf{loc}}_{k}$ and $\operatorname{\textsf{scd}}_{k}$, for constant $k$, the polynomial time solvability of $\operatorname{\textsc{Match}}$ does not only extend from $\mathcal{C}$ to exact repetitions, but also to such skeleton-repetitions, called $\mathcal{C}$-repetitions.

Theorem 14** ([16]).**

For $\mathcal{C}\in\{\operatorname{\textsf{nc}},\operatorname{\textsf{reg}},\operatorname{\textsf{loc}}_{k},\operatorname{\textsf{scd}}_{k}\}$, solving the matching problem for the class of $\mathcal{C}$-repetitions can be done in polynomial time.

It is interesting to note that the general treewidth-based framework of polynomial time matching of patterns does not seem to cover a very simple and natural aspect: repetitions of the same pattern. More precisely, if $\mathcal{C}$ is one of the known efficiently matchable classes of patterns, then a repetition $\alpha^{k}$ for some $\alpha\in\mathcal{C}$ is usually not in $\mathcal{C}$ anymore. In fact, it can be shown that even for patterns $\alpha$ with bounded and very low treewidth, the treewidth of repetitions $\alpha^{k}$ can be unbounded.

Theorem 15** ([16]).**

Let ${\mathcal{C}}$ be a class of patterns that contains $\operatorname{\textsf{reg}}$. Then the class of ${\mathcal{C}}$-repetitions contains patterns with arbitrarily large treewidth.

In particular, the previous theorem holds for the class $\operatorname{\textsf{reg}}$ of regular patterns, arguably the simplest class allowing an unbounded number of variables (note that patterns with a constant number of variables can trivially be matched in polynomial-time). In the same paper it is shown that if the notion of repetition is relaxed further, by considering a setting where the order in which the variables appear is no longer constrained at all (i.e., considering abelian repetitions instead of repetitions), then the matching problem is $\operatorname{\mathsf{NP}}$-complete. This holds even in the minimal case when the number of repetitions is restricted to two, and that the pattern which is repeated is regular.

7 From Locality to Graph Parameters

Following the ideas of Section 3 we explore further the connection between string and graph parameters. The main idea behind such a connection is to reach it by “flattening” a graph into a sequential form, or by “inflating” a string into a graph, so that algorithmic techniques available for each one of these become applicable for the other one as well. In this section, following [13], we are concerned with certain structural parameters (and the problems of computing them) for graphs and strings: the cutwidth $\operatorname{\textsf{cw}}(G)$ of a graph $G$ (i. e., the maximum number of “stacked” edges if the vertices of a graph are drawn on a straight line), the pathwidth $\operatorname{\textsf{pw}}(G)$ of a graph $G$ (i. e., the minimum width of a tree decomposition the tree structure of which is a path), and the locality number $\operatorname{\textsf{loc}}(\alpha)$ of a string $\alpha$ (explained in more detail in Section 4.2). By Cutwidth, Pathwidth and Loc, we denote the corresponding natural decision problems (i. e., decide whether a given graph has a pathwidth/cutwidth, or a given string has a locality number of at most $k$, for given $k$) and with the prefix Min, we refer to the minimisation variants. The two former graph-parameters are very classical. Pathwidth is a simple (yet still hard to compute) subvariant of treewidth, which measures how much a graph resembles a path. The problems Pathwidth and MinPathwidth are intensively studied (in terms of exact, parameterised and approximation algorithms) and have numerous applications (see the surveys and textbook [9, 54, 7]). Cutwidth is the best known example of a whole class of so-called graph layout problems (see the survey [20, 71] for detailed information), which are studied since the 1970s and were originally motivated by questions of circuit layouts.

In comparison, the locality number seems a rather simple parameter directly defined on strings, but, however, it bounds the treewidth of the string (in the sense defined in Section 4.1), and the corresponding marking sequences can be seen as instructions for a dynamic programming algorithm for matching the pattern. In this way, it resembles a bit to the way the pathwidth and treewidth of graphs are used in algorithmic settings. Moreover, compared to other “tractability-parameters” of strings, it seems to cover best the treewidth of a string, but it also cannot be efficiently computed compared to the other simpler parameters.

Going more into detail, for Loc, exact exponential-time algorithms are not hard to be devised [15] but whether it can be solved in polynomial-time, or whether it is at least fixed-parameter tractable was left open in the paper where this measure was introduced. On the other hand, Pathwidth and Cutwidth are known $\operatorname{\mathsf{NP}}$-complete problems, fixed-parameter tractable with respect to parameter $\operatorname{\textsf{pw}}(G)$ or $\operatorname{\textsf{cw}}(G)$, respectively (even with “linear” fpt-algorithms with running-time $g(k)\operatorname{O}(n)$ [8, 10, 80]). With respect to approximation, their minimisation variants have received a lot of attention, mainly because they yield (like many other graph parameters) general algorithmic approaches for numerous graph problems, i. e., a good linear arrangement or path-decomposition can often be used to design a dynamic programming (or even divide and conquer) algorithm for other problems. The best known approximation algorithms for the problems MinPathwidth and MinCutwidth (with approximations ratios of $\operatorname{O}(\sqrt{\log(\operatorname{\textsf{opt}})}\log(n))$ and $\operatorname{O}(\log^{2}(n))$, respectively) follow from approximations of vertex separators (see [25]) and edge separators (see [58]), respectively.

There are two natural approaches to represent a word $\alpha$ over alphabet $\Sigma$ as a graph $G_{\alpha}=(V_{\alpha},E_{\alpha})$: (1) $V_{\alpha}=\{1,2,\ldots,|\alpha|\}$ and the edges are somehow used to represent the actual symbols (note that this is the case for the standard graph representation of patterns defined in Section 4.1), or (2) $V_{\alpha}=\Sigma$ and the edges are somehow used to represent the positions of $\alpha$. A reduction of type (2) can be defined such that $|E_{\alpha}|=\operatorname{O}(|\alpha|)$ and $\operatorname{\textsf{cw}}(G_{\alpha})=2\operatorname{\textsf{loc}}(\alpha)$, and a reduction of type (1) can be defined such that $|E_{\alpha}|=\operatorname{O}(|\alpha|^{2})$ and $\operatorname{\textsf{loc}}(\alpha)\leq\operatorname{\textsf{pw}}(G_{\alpha})\leq 2\operatorname{\textsf{loc}}(\alpha)$. Since these reductions are parameterised reductions and also allow to transfer approximation results, one may conclude that Loc is fixed-parameter tractable if parameterised by $|\Sigma|$ (note that for parameter $|\Sigma|$ a simple, but less efficient fpt-algorithm is trivially obtained by simply enumerating all marking sequences) or by the locality number, and also that there is a polynomial-time $\operatorname{O}(\sqrt{\log(\operatorname{\textsf{opt}})}\log(n))$-approximation algorithm for MinLoc.

In addition, one can represent an arbitrary multi-graph $G=(V,E)$ by a word $\alpha_{G}$ over alphabet $V$ with $|\alpha_{G}|=|E|$ and $\operatorname{\textsf{cw}}(G)=\operatorname{\textsf{loc}}(\alpha)$. This describes a Turing-reduction from Cutwidth to Loc which also allows to transfer approximation results between the minimisation variants. As a result, Loc is $\operatorname{\mathsf{NP}}$-complete. Finally, by plugging together the reductions from MinCutwidth to MinLoc and from MinLoc to MinPathwidth, one obtains a reduction which transfers approximation results from MinPathwidth to MinCutwidth, which yields an $\operatorname{O}(\sqrt{\log(\operatorname{\textsf{opt}})}\log(n))$-approximation algorithm for MinCutwidth. This result from [13] improved, for the first time since 1999, the best approximation for Cutwidth from [58]. Interestingly, this improvement appeared as a side-product of relating string-parameters with graph-parameters.

Theorem 16** ([13]).**

There is an $\operatorname{O}(\sqrt{\log(\operatorname{\textsf{opt}})}\log(h))$-approximation algorithm (running in polynomial time) for MinCutwidth on multigraphs with $h$ edges. In particular, this yields an $\operatorname{O}(\sqrt{\log(\operatorname{\textsf{opt}})}\log(n))$-approximation algorithm for MinCutwidth for graphs.

Moreover, this approach allows also for establishing a direct connection between cutwidth and pathwidth, which preserves the good algorithmic properties, and has not yet been reported in the literature so far. This is rather surprising, since Cutwidth and Pathwidth have been jointly investigated in the context of exact and approximation algorithms, especially in terms of balanced vertex and edge separators. We think that a reason for overlooking this connection might be that it is less obvious on the graph level and becomes more apparent if linked via the string parameter of locality, emphasising, as such, the value of such mixed approaches.

8 Extensions

8.1 Injectivity

In our setting, the substitutions that map variables to words are not required to be injective, i. e., different variables can be mapped to the same word. However, the requirement of injectivity is natural in some contexts. For example, in the pattern matching community, the first mentioning of pattern matching with variables concerns the case where variables have to be substituted by single symbols and in an injective way. More precisely, this parameterised pattern matching was introduced in [3] to formalise the problem of detecting code clones (i. e., we want to find code segments that are created by copying some code blocks and renaming program variables (this renaming will be injective, since otherwise the semantic of the code might change)). More generally speaking, the injectivity condition is appropriate whenever we know a priori that different variables should always refer to different words (e. g., when matching the pattern

[TABLE]

in order to check whether there is a repetition of some name-address data tuple, then it is likely that we can assume injectivity).

Depending on the actual variant, the injectivity condition can make the matching problem harder or easier. In [26], it is shown that it is $\operatorname{\mathsf{NP}}$-hard to decide for a given word $w$ and an integer $k$ whether $w$ can be factorised into at least $k$ pairwise different factors. This immediately implies that the injectivity condition makes the matching problem $\operatorname{\mathsf{NP}}$-hard even for the “trivial” pattern class $\{x_{1}x_{2}\ldots x_{n}\mid n\geq 1\}$ (note that this is even a subset of the class $\operatorname{\textsf{reg}}$ of regular patterns). On the other hand, if we have an upper bound on $|\Sigma|$ and $\max\{|h(x)|\mid x\in X\}$ (recall that this case is still $\operatorname{\mathsf{NP}}$-hard even for bounds $2$ and $1$, respectively; see Theorem 2) then also the total number of possible substitution words is bounded; thus, the injectivity condition bounds the total number of variables and therefore the matching problem becomes tractable (see [28]). A similar observation can be made with respect to fixed-parameter tractability if we parameterise by $|\Sigma|$ and $\max\{|h(x)|\mid x\in X\}$ (see [29]).

8.2 Word Equations

A word equation is an equality $\alpha=\beta$, where $\alpha$ and $\beta$ are patterns with variables, e. g., $\alpha=x_{1}\mathtt{a}\mathtt{b}x_{2}$ and $\beta=\mathtt{a}x_{1}x_{2}\mathtt{b}$ define the equation $x_{1}\mathtt{a}\mathtt{b}x_{2}=\mathtt{a}x_{1}x_{2}\mathtt{b}$. A solution to an equation $\alpha=\beta$ is a substitution $h:(\operatorname{\textsf{var}}(\alpha)\cup\operatorname{\textsf{var}}(\beta))\rightarrow\Sigma^{*}$ (in the sense defined in Section 2) that satisfies $h(\alpha)=h(\beta)$. For the example equation from above, the solutions are the substitutions $h$ with $h(x_{1})=\mathtt{a}^{k}$, for $k\geq 0$, and $h(x_{2})=\mathtt{b}^{\ell}$, for $\ell\geq 0$.

The study of word equations (or the existential theory of equations over free monoids) is an important topic found at the intersection of algebra and computer science, with significant connections to, e.g., combinatorial group or monoid theory [64, 63, 21], unification [78, 46, 47]), and, more recently, data base theory [33, 32].

The central computational problem for word equations is the satisfiability problem, i. e., the problem of deciding whether a given word equation $\alpha=\beta$ has a solution or not. In this regard, the matching problem for patterns with variables describes just the special case of the satisfiability problem for word equations where one side of the equation is a terminal word, e. g., $x_{1}\mathtt{a}\mathtt{b}x_{1}x_{2}\mathtt{c}x_{2}x_{1}=\mathtt{b}\mathtt{a}\mathtt{b}\mathtt{b}\mathtt{a}\mathtt{c}\mathtt{a}\mathtt{b}$ is an instance of the matching problem already mentioned in the introduction, phrased as a word equation. Consequently, the satisfiability problem is intractable, even for very strongly restricted cases (see Theorems 2 and 3). Also note that it has been shown in [22] that the solvability problem remains $\operatorname{\mathsf{NP}}$-hard if every variable has at most two occurrences in $\alpha\beta$ (called quadratic equations), but the proof of [22] actually talks about the matching problem for patterns with at most two occurrences per variable.

While the matching problem for patterns with variables is trivially decidable, it is not at all obvious how to solve the satisfiability problem for word equations. In fact, the question whether it is decidable was initially approached with the expectation that it will be answered in the negative. It was, however, shown to be decidable by Makanin [65] (see Chapter $12$ of [62] for a survey). Later it was shown that the satisfiability problem is in $\operatorname{\mathsf{PSPACE}}$ by Plandowski [72]; a new proof of this result was obtained in [49], based on a new simple technique called recompression. There are also cases when the satisfiability problem is tractable. For instance, word equations with only one variable can be solved in linear time in the size of the equation, see [48]; equations with two variables can be solved in time $\operatorname{O}(|\alpha\beta|^{5})$, see [19].

Given the fact that there are many structural restrictions of patterns that yield tractability (with respect to the matching problem, see Section 4), the question naturally arises how the complexity of the satisfiability problem for word equations (which are essentially equations of patterns) behaves if these restrictions are applied to word equations. More precisely, while each class of patterns with $\operatorname{\mathsf{NP}}$-hard matching problem yields a class of word equations with $\operatorname{\mathsf{NP}}$-hard satisfiability problem, the hardness of the satisfiability problem for equations with sides in some efficiently matchable class of patterns is no longer immediate. An investigation of that question was initiated in [66], where the following results were obtained. Firstly, the satisfiability problem for non-cross word equations (i. e., word equations for which both sides are non-cross) remains $\operatorname{\mathsf{NP}}$-hard. In particular, solving non-cross equations $\alpha=\beta$ where each variable occurs at most three times, at most twice in $\alpha$ and exactly once in $\beta$, is $\operatorname{\mathsf{NP}}$-hard (note that this constitutes the first $\operatorname{\mathsf{NP}}$-hardness result for word equations that is not a direct conclusion from a hardness result for the matching problem). Secondly, the satisfiability of one-repeated variable equations (i. e., at most one variable occurs more than once in $\alpha\beta$, but arbitrarily many other variables occur only once) having at least one non-repeated variable on each side, was shown to be trivially in $\operatorname{\mathsf{P}}$.

In [18], it is shown that it is (still) $\operatorname{\mathsf{NP}}$-hard to solve regular ordered word equations. More precisely, these are word equations where each side is a regular pattern and the order of the variables in both sides is the same (it is, however, possible that some variables only occur on one side of the equation), e. g., $x_{1}\mathtt{a}x_{2}\mathtt{b}\mathtt{a}x_{3}x_{4}=\mathtt{b}x_{1}x_{3}\mathtt{a}\mathtt{a}x_{4}$ is a regular ordered word equation. They are particular cases of both quadratic equations and non-cross equations, so the reductions showing the hardness of solving these more general equations do not carry over. In particular, note that the class of regular patterns is arguably the most simple class of patterns in terms of their matching complexity (see Section 5.1).

The respective hardness reduction relied on some deep word-combinatorics ideas. As a first step, a reachability problem for a certain type of (regulated) string rewriting systems was introduced, and showed it is $\operatorname{\mathsf{NP}}$-complete. This was achieved via a reduction from the problem 3-Partition [38], which is strongly $\operatorname{\mathsf{NP}}$-complete. Then it was shown that this reachability problem can be reduced to the satisfiability of regular-ordered word equations; in this reduction the applications of the rewriting rules of the system were encoded into the periods of the words assigned to the variables in a solution to the equation. The main technicality was to make sure to only use one occurrence of each variable per side, and moreover to even have the variables in the same order in both sides. This result exhibits the arguably structurally-simplest class of word equations for which the satisfiability problem is $\operatorname{\mathsf{NP}}$-hard.

The main open problem in the area of word equations remains, even for simple subclasses such as regular equations or quadratic equations, to show that the satisfiability problem of word equations of the respective types is in $\operatorname{\mathsf{NP}}$ (note that this was already explicitly posed as an open question for the class of quadratic word equations in [22]).

9 Conclusions

In this work we tried to survey several results related to the problem of matching patterns with variables, that seem important to us. While this work is clearly not exhaustive, it is aimed to offer a basic understanding of the problems and state of the art in this area.

From an algorithmic point of view, the results we covered provide a wide variety of classes of patterns with variables, for which $\operatorname{\textsc{Match}}$ can be efficiently solved. Moreover, as explained in Section 4.3, it is usually easy to check whether a pattern belongs to one of these classes. So, putting it all together, one could use the following approach when trying to match a pattern, rather than just using an exponential time algorithm (based, e.g., on general SMT-solvers, or on the theory of string solving [6, 81]). First, check if the pattern belongs to one of the classes for which efficient matching algorithms are known and, then, use this algorithm; only use a general algorithm when no customised one can be applied. Identifying more natural pattern classes for which $\operatorname{\textsc{Match}}$ can be solved efficiently appears, as such, as a rather useful task. Following the practically motivated challenges that arise from the area of string solving, one could also try to find efficient matching algorithms for various classes of patterns, enhanced with various constraints: regular constraints, length constraints, etc.

As an important part of this survey deals with polynomial time algorithms, it is natural to also ask whether they are optimal or not. This kind of questions are the focus of the area of fine-grained complexity (see, e.g., the survey [11] and the citations therein). It would be interesting to see, using tools from this area, whether one can show lower bounds for the $\operatorname{\textsc{Match}}$ problems for different classes of patterns.

In the light of the results from [13], it seems that exploring the connections between string parameters and parameters for other classes of objects could lead to some interesting results in both worlds. So, it also seems like an interesting challenge to explore what the structural parameters of strings that we explored here (and maybe some other new ones) mean when various other types of data are represented as strings, and what consequences can be derived from such a representation.

Finally, the area of word equations abounds with open problems. As mentioned, it is not even clear whether the satisfiability of regular or quadratic equations is in $\operatorname{\mathsf{NP}}$. So even if we restrict to equations with structurally simple left and right hand sides, the complexity of solving equations is not known. Such problems become even more involved when we consider equations with various types of constraints (e.g., length or regular). For instance, the decidability of general word equations with length constraints is a long standing open problem, but it is already an interesting open question for simpler cases (once again: regular or quadratic equations); see, e.g., [44, 17, 59], and the references therein. It seems interesting to us whether some of the ideas used in matching patterns can be transferred to solving (simplified) word equations, with or without constraints.