Efficient $k$-NN Search in IoT Data: Overlap Optimization in Tree-Based Indexing Structures

TL;DR

This paper introduces three heuristics to reduce data space overlap in tree-based indexes for IoT data, significantly improving search efficiency and system scalability.

Contribution

It presents novel overlap reduction heuristics—volume-based, distance-based, and object-based—for optimizing tree index construction in IoT data management.

Findings

Reduced search time through overlap minimization

Improved index construction efficiency

Enhanced system scalability

Abstract

The proliferation of interconnected devices in the Internet of Things (IoT) has led to an exponential increase in data, commonly known as Big IoT Data. Efficient retrieval of this heterogeneous data demands a robust indexing mechanism for effective organization. However, a significant challenge remains: the overlap in data space partitions during index construction. This overlap increases node access during search and retrieval, resulting in higher resource consumption, performance bottlenecks, and impedes system scalability. To address this issue, we propose three innovative heuristics designed to quantify and strategically reduce data space partition overlap. The volume-based method (VBM) offers a detailed assessment by calculating the intersection volume between partitions, providing deeper insights into spatial relationships. The distance-based method (DBM) enhances efficiency by using the distance between partition centers and radii to evaluate overlap, offering a streamlined yet accurate approach. Finally, the object-based method (OBM) provides a practical solution by counting objects across multiple partitions, delivering an intuitive understanding of data space dynamics. Experimental results demonstrate the effectiveness of these methods in reducing search time, underscoring their potential to improve data space partitioning and enhance overall system performance.