Hammer: Towards Efficient Hot-Cold Data Identification via Online Learning

TL;DR

This paper introduces Hammer, an online learning-based method for accurately and efficiently classifying data as hot or cold in dynamic storage environments, outperforming traditional approaches in accuracy and overhead.

Contribution

Hammer is the first online learning approach for hot-cold data identification that adapts to changing workloads with high accuracy and low operational costs.

Findings

Achieves 90% accuracy in hot-cold classification.

Reduces computational and storage overheads significantly.

Demonstrates effectiveness on synthetic and real-world datasets.

Abstract

Efficient management of storage resources in big data and cloud computing environments requires accurate identification of data's "cold" and "hot" states. Traditional methods, such as rule-based algorithms and early AI techniques, often struggle with dynamic workloads, leading to low accuracy, poor adaptability, and high operational overhead. To address these issues, we propose a novel solution based on online learning strategies. Our approach dynamically adapts to changing data access patterns, achieving higher accuracy and lower operational costs. Rigorous testing with both synthetic and real-world datasets demonstrates a significant improvement, achieving a 90% accuracy rate in hot-cold classification. Additionally, the computational and storage overheads are considerably reduced.