# Fast Approximation of Frequent $k$-mers and Applications to Metagenomics

## TL;DR

SAKEIMA is a sampling-based method that efficiently approximates frequent $k$-mers in large sequencing datasets with rigorous guarantees, enabling faster biological analyses without processing entire datasets.

## Contribution

This work introduces SAKEIMA, a novel sampling approach that uses VC dimension analysis to accurately estimate frequent $k$-mers with fewer data processed.

## Key findings

- SAKEIMA achieves significant speed-ups in large dataset analysis.
- It provides rigorous approximation guarantees for $k$-mer frequencies.
- Accurate $k$-mer based distances are computed using only a fraction of the data.

## Abstract

Estimating the abundances of all $k$-mers in a set of biological sequences is a fundamental and challenging problem with many applications in biological analysis. While several methods have been designed for the exact or approximate solution of this problem, they all require to process the entire dataset, that can be extremely expensive for high-throughput sequencing datasets. While in some applications it is crucial to estimate all $k$-mers and their abundances, in other situations reporting only frequent $k$-mers, that appear with relatively high frequency in a dataset, may suffice. This is the case, for example, in the computation of $k$-mers' abundance-based distances among datasets of reads, commonly used in metagenomic analyses. In this work, we develop, analyze, and test, a sampling-based approach, called SAKEIMA, to approximate the frequent $k$-mers and their frequencies in a high-throughput sequencing dataset while providing rigorous guarantees on the quality of the approximation. SAKEIMA employs an advanced sampling scheme and we show how the characterization of the VC dimension, a core concept from statistical learning theory, of a properly defined set of functions leads to practical bounds on the sample size required for a rigorous approximation. Our experimental evaluation shows that SAKEIMA allows to rigorously approximate frequent $k$-mers by processing only a fraction of a dataset and that the frequencies estimated by SAKEIMA lead to accurate estimates of $k$-mer based distances between high-throughput sequencing datasets. Overall, SAKEIMA is an efficient and rigorous tool to estimate $k$-mers abundances providing significant speed-ups in the analysis of large sequencing datasets.