Normalized Google Distance of Multisets with Applications

TL;DR

This paper introduces a new normalized Google distance for multisets of search terms, providing a more effective semantic measure for applications, based on Kolmogorov complexity, and compares it with pairwise NGD results.

Contribution

It proposes a novel NGD for multisets of terms, enhancing semantic analysis capabilities beyond pairwise comparisons, with applications demonstrated and evaluated.

Findings

Multiset NGD improves semantic similarity measurement.

Compared multiset NGD results with pairwise NGD, showing better application performance.

Demonstrated applications in various semantic analysis tasks.

Abstract

Normalized Google distance (NGD) is a relative semantic distance based on the World Wide Web (or any other large electronic database, for instance Wikipedia) and a search engine that returns aggregate page counts. The earlier NGD between pairs of search terms (including phrases) is not sufficient for all applications. We propose an NGD of finite multisets of search terms that is better for many applications. This gives a relative semantics shared by a multiset of search terms. We give applications and compare the results with those obtained using the pairwise NGD. The derivation of NGD method is based on Kolmogorov complexity.