Hierarchical One Permutation Hashing: Efficient Multimedia Near Duplicate Detection

TL;DR

This paper introduces Hierarchical One Permutation Hashing (HOPH), a novel, efficient hashing method for multimedia near duplicate detection that significantly reduces preprocessing and comparison times while maintaining accuracy.

Contribution

The paper proposes HOPH, a new hierarchical hashing technique that improves efficiency in multimedia duplicate detection over existing methods like minwise hashing.

Findings

HOPH is 5-7 times faster than existing methods.

HOPH maintains similar accuracy to traditional hashing methods.

Experiments on real datasets validate the efficiency gains.

Abstract

With advances in multimedia technologies and the proliferation of smart phone, digital cameras, storage devices, there are a rapidly growing massive amount of multimedia data collected in many applications such as multimedia retrieval and management system, in which the data element is composed of text, image, video and audio. Consequently, the study of multimedia near duplicate detection has attracted significant concern from research organizations and commercial communities. Traditional solution minwish hashing (\minwise) faces two challenges: expensive preprocessing time and lower comparison speed. Thus, this work first introduce a hashing method called one permutation hashing (\oph) to shun the costly preprocessing time. Based on \oph, a more efficient strategy group based one permutation hashing (\goph) is developed to deal with the high comparison time. Based on the fact that the similarity of most multimedia data is not very high, this work design an new hashing method namely hierarchical one permutation hashing (\hoph) to further improve the performance. Comprehensive experiments on real multimedia datasets clearly show that with similar accuracy \hoph is five to seven times faster than