Differentially Private Motif-Preserving Multi-modal Hashing

TL;DR

This paper introduces DMP-MH, a privacy-preserving multi-modal hashing method that protects sensitive relational motifs in cross-modal retrieval graphs while maintaining high accuracy.

Contribution

It proposes a novel framework that bounds motif sensitivity and generates a sanitized graph, enabling effective private cross-modal hashing with strong utility.

Findings

DMP-MH outperforms private baselines by up to 11.4 mAP points.

It retains up to 92.5% of non-private performance.

The approach effectively protects relational motifs against link reconstruction attacks.

Abstract

Cross-modal hashing enables efficient retrieval by encoding images and text into compact binary codes. State-of-the-art methods rely on semantic similarity graphs derived from user interactions for supervision, yet these graphs encode sensitive behavioral patterns vulnerable to link reconstruction attacks. Existing privacy-preserving approaches fail on graph-structured data: Differentially Private SGD destroys relational motifs by treating samples independently, while graph synthesis methods suffer from unbounded local sensitivity in scale-free networks, hub nodes cause single-edge modifications to alter triangle counts by $\mathcal{O}(N)$, necessitating prohibitive noise injection. We term this phenomenon Hubness Explosion. We propose DMP-MH, a Sanitize-then-Distill framework that decouples privacy from representation learning. Our approach first bounds sensitivity by deterministically clipping node degrees, capping the $L_2$-sensitivity of triangle motifs independently of dataset size. A sanitized synthetic graph is then generated via Noisy Mirror Descent under $(\epsilon,\delta)$-Edge Differential Privacy. Finally, dual-stream hashing networks distill this topology using a holistic structural loss that enforces cross-modal alignment. Evaluated on MIRFlickr-25K and NUS-WIDE under a strict inductive protocol, DMP-MH outperforms private baselines by up to 11.4 mAP points while retaining up to 92.5% of non-private performance.