Handling Overload Conditions In High Performance Trustworthy Information Retrieval Systems

TL;DR

This paper introduces an optimal load shedding algorithm designed to maintain trustworthy and high-quality web search results with minimal response time during overload conditions.

Contribution

It presents a novel load shedding algorithm specifically adapted for real-time data stream applications in web search engines to handle overloads effectively.

Findings

Enables trustworthy search results during overloads

Maintains optimal response time under high load

Improves reliability of web search systems

Abstract

Web search engines retrieve a vast amount of information for a given search query. But the user needs only trustworthy and high-quality information from this vast retrieved data. The response time of the search engine must be a minimum value in order to satisfy the user. An optimum level of response time should be maintained even when the system is overloaded. This paper proposes an optimal Load Shedding algorithm which is used to handle overload conditions in real-time data stream applications and is adapted to the Information Retrieval System of a web search engine. Experiment results show that the proposed algorithm enables a web search engine to provide trustworthy search results to the user within an optimum response time, even during overload conditions.