DWBench: Holistic Evaluation of Watermark for Dataset Copyright Auditing
Xiao Ren, Xinyi Yu, Linkang Du, Min Chen, Yuanchao Shu, Zhou Su, Yunjun Gao, Zhikun Zhang

TL;DR
This paper introduces DWBench, a comprehensive benchmark and framework for evaluating watermarks in datasets used for deep learning, addressing inconsistencies and enabling fair comparison of watermarking methods across tasks.
Contribution
The paper proposes a structured taxonomy for watermarking methods and develops DWBench, an open-source toolkit for systematic evaluation of dataset watermark techniques in classification and generation tasks.
Findings
No single watermarking method outperforms others in all scenarios.
Classification and generation tasks require different watermarking approaches.
Existing techniques are unstable at low watermark rates and in multi-user environments.
Abstract
The surging demand for large-scale datasets in deep learning has heightened the need for effective copyright protection, given the risks of unauthorized use to data owners. Although the dataset watermark technique holds promise for auditing and verifying usage, existing methods are hindered by inconsistent evaluations, which impede fair comparisons and assessments of real-world viability. To address this gap, we propose a two-layer taxonomy that categorizes methods by implementation (model-based vs. model-free injection; model-behavior vs. model-message verification), offering a structured framework for cross-task analysis. Then, we develop DWBench, a unified benchmark and open-source toolkit for systematically evaluating image dataset watermark techniques in classification and generation tasks. Using DWBench, we assess 25 representative methods under standardized conditions,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI) · Advanced Neural Network Applications
