Totems: Physical Objects for Verifying Visual Integrity
Jingwei Ma, Lucy Chai, Minyoung Huh, Tongzhou Wang, Ser-Nam Lim,, Phillip Isola, Antonio Torralba

TL;DR
This paper proposes a novel image forensic method using physical objects called totems that distort light to embed scene verification cues, making manipulation detection more robust without training on manipulation datasets.
Contribution
Introduction of totems as physical scene markers for verifying image integrity, leveraging geometric and material properties instead of learning-based models.
Findings
Totems enable detection of image manipulations through scene inconsistencies.
The method does not require training data, relying on physical properties.
It increases difficulty for adversaries to manipulate images without detection.
Abstract
We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems bend and redirect light rays, thus providing multiple, albeit distorted, views of the scene within a single image. A defender can use these distorted totem pixels to detect if an image has been manipulated. Our approach unscrambles the light rays passing through the totems by estimating their positions in the scene and using their known geometric and material properties. To verify a totem-protected image, we detect inconsistencies between the scene reconstructed from totem viewpoints and the scene's appearance from the camera viewpoint. Such an approach makes the adversarial manipulation task more difficult, as the adversary must modify both the totem and image pixels in a geometrically consistent manner…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Media Forensic Detection · Adversarial Robustness in Machine Learning · Integrated Circuits and Semiconductor Failure Analysis
