Good parts first - a new algorithm for approximate search in lexica and string databases

TL;DR

This paper introduces a new efficient algorithm for approximate search in lexica and string databases, enabling fast retrieval of similar entries with large error bounds by organizing searches around exact partial matches and extending them stepwise.

Contribution

The paper presents a novel search algorithm that improves efficiency by starting with exact partial matches and extending them, supporting large error bounds and flexible starting points.

Findings

Significant efficiency improvements over existing methods

Supports large error bounds for approximate search

Effective for long strings like phrases and sentences

Abstract

We present a new efficient method for approximate search in electronic lexica. Given an input string (the pattern) and a similarity threshold, the algorithm retrieves all entries of the lexicon that are sufficiently similar to the pattern. Search is organized in subsearches that always start with an exact partial match where a substring of the input pattern is aligned with a substring of a lexicon word. Afterwards this partial match is extended stepwise to larger substrings. For aligning further parts of the pattern with corresponding parts of lexicon entries, more errors are tolerated at each subsequent step. For supporting this alignment order, which may start at any part of the pattern, the lexicon is represented as a structure that enables immediate access to any substring of a lexicon word and permits the extension of such substrings in both directions. Experimental evaluations of the approximate search procedure are given that show significant efficiency improvements compared to existing techniques. Since the technique can be used for large error bounds it offers interesting possibilities for approximate search in special collections of "long" strings, such as phrases, sentences, or book ti