Resilient Pattern Mining

TL;DR

This paper introduces the $( au, k)$-Resilient Pattern Mining problem, proposing an efficient algorithm to find substrings resilient to substitutions, with applications in genomic data analysis and clustering.

Contribution

The paper defines a new resilient pattern mining problem and provides an exact, efficient algorithm with practical experiments demonstrating its usefulness.

Findings

Resilient substrings outperform frequent substrings in genomic analysis.

The proposed algorithm is significantly faster and more space-efficient than baseline methods.

Clustering based on resilient substrings is effective for large datasets.

Abstract

Frequent pattern mining is a flagship problem in data mining. In its most basic form, it asks for the set of substrings of a given string $S$ of length $n$ that occur at least $\tau$ times in $S$, for some integer $\tau\in[1,n]$. We introduce a resilient version of this classic problem, which we term the $(\tau, k)$-Resilient Pattern Mining (RPM) problem. Given a string $S$ of length $n$ and two integers $\tau, k\in[1,n]$, RPM asks for the set of substrings of $S$ that occur at least $\tau$ times in $S$, even when the letters at any $k$ positions of $S$ are substituted by other letters. Unlike frequent substrings, resilient ones account for the fact that changes to string $S$ are often expensive to handle or are unknown. We propose an exact $\mathcal{O}(n\log n)$-time and $\mathcal{O}(n)$-space algorithm for RPM, which employs advanced data structures and combinatorial insights. We then present experiments on real large-scale datasets from different domains demonstrating that: (I) The notion of resilient substrings is useful in analyzing genomic data and is more powerful than that of frequent substrings, in scenarios where resilience is required, such as in the case of versioned datasets; (II) Our algorithm is several orders of magnitude faster and more space-efficient than a baseline algorithm that is based on dynamic programming; and (III) Clustering based on resilient substrings is effective.