Search in the Universe of Big Networks and Data

TL;DR

This paper analyzes the efficiency of search strategies in infinite data spaces, considering multiple agents and randomness, and provides formulas for average search time and energy consumption, highlighting the role of time-outs.

Contribution

It introduces a general framework for understanding search in large networks, deriving formulas for search time and energy, and explores the impact of multiple agents and adversarial conditions.

Findings

Search can be successful in infinite spaces with random search and time-outs.

Formulas for average search time and energy consumption are derived.

Optimal number of search agents can be estimated to guarantee timely discovery.

Abstract

Searching in the Internet for some object characterised by its attributes in the form of data, such as a hotel in a certain city whose price is less than something, is one of our most common activities when we access the Web. We discuss this problem in a general setting, and compute the average amount of time and the energy it takes to find an object in an infinitely large search space. We consider the use of N search agents which act concurrently. Both the case where the search agent knows which way it needs to go to find the object, and the case where the search agent is perfectly ignorant and may even head away from the object being sought. We show that under mild conditions regarding the randomness of the search and the use of a time-out, the search agent will always find the object despite the fact that the search space is infinite. We obtain a formula for the average search time and the average energy expended by N search agents acting concurrently and independently of each other. We see that the time-out itself can be used to minimise the search time and the amount of energy that is consumed to find an object. An approximate formula is derived for the number of search agents that can help us guarantee that an object is found in a given time, and we discuss how the competition between search agents and other agents that try to hide the data object, can be used by opposing parties to guarantee their own success.