Evolutionary Algorithms Approach For Search Based On Semantic Document Similarity

TL;DR

This paper presents a novel search method combining semantic embeddings from the Universal Sentence Encoder with evolutionary algorithms like Genetic and Differential Evolution to improve document retrieval accuracy.

Contribution

It introduces a transfer learning-based approach integrating USE with evolutionary algorithms for semantic search, demonstrating improved retrieval performance on the SQuAD dataset.

Findings

Evolutionary algorithms outperform traditional ranking methods in top N document retrieval.

USE effectively captures semantic similarity in text documents.

Genetic and Differential Evolution algorithms efficiently identify relevant documents.

Abstract

Advancements in cloud computing and distributed computing have fostered research activities in Computer science. As a result, researchers have made significant progress in Neural Networks, Evolutionary Computing Algorithms like Genetic, and Differential evolution algorithms. These algorithms are used to develop clustering, recommendation, and question-and-answering systems using various text representation and similarity measurement techniques. In this research paper, Universal Sentence Encoder (USE) is used to capture the semantic similarity of text; And the transfer learning technique is used to apply Genetic Algorithm (GA) and Differential Evolution (DE) algorithms to search and retrieve relevant top N documents based on user query. The proposed approach is applied to the Stanford Question and Answer (SQuAD) Dataset to identify a user query. Finally, through experiments, we prove that text documents can be efficiently represented as sentence embedding vectors using USE to capture the semantic similarity, and by comparing the results of the Manhattan Distance, GA, and DE algorithms we prove that the evolutionary algorithms are good at finding the top N results than the traditional ranking approach.