Search Using N-gram Technique Based Statistical Analysis for Knowledge Extraction in Case Based Reasoning Systems

TL;DR

This paper introduces a novel N-gram based statistical method for knowledge extraction in Case Based Reasoning systems, utilizing transformations, noise removal, and probability analysis to improve matching accuracy.

Contribution

It presents a new N-gram comparison technique combined with error elimination and probability analysis for enhanced case matching in reasoning systems.

Findings

Improved accuracy in case matching using N-gram analysis

Effective noise removal enhances data quality

Customizable threshold for match sensitivity

Abstract

Searching techniques for Case Based Reasoning systems involve extensive methods of elimination. In this paper, we look at a new method of arriving at the right solution by performing a series of transformations upon the data. These involve N-gram based comparison and deduction of the input data with the case data, using Morphemes and Phonemes as the deciding parameters. A similar technique for eliminating possible errors using a noise removal function is performed. The error tracking and elimination is performed through a statistical analysis of obtained data, where the entire data set is analyzed as sub-categories of various etymological derivatives. A probability analysis for the closest match is then performed, which yields the final expression. This final expression is referred to the Case Base. The output is redirected through an Expert System based on best possible match. The threshold for the match is customizable, and could be set by the Knowledge-Architect.