Totems: Physical Objects for Verifying Visual Integrity

TL;DR

This paper proposes a novel image forensic method using physical objects called totems that distort light to embed scene verification cues, making manipulation detection more robust without training on manipulation datasets.

Contribution

Introduction of totems as physical scene markers for verifying image integrity, leveraging geometric and material properties instead of learning-based models.

Findings

Totems enable detection of image manipulations through scene inconsistencies.

The method does not require training data, relying on physical properties.

It increases difficulty for adversaries to manipulate images without detection.

Abstract

We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems bend and redirect light rays, thus providing multiple, albeit distorted, views of the scene within a single image. A defender can use these distorted totem pixels to detect if an image has been manipulated. Our approach unscrambles the light rays passing through the totems by estimating their positions in the scene and using their known geometric and material properties. To verify a totem-protected image, we detect inconsistencies between the scene reconstructed from totem viewpoints and the scene's appearance from the camera viewpoint. Such an approach makes the adversarial manipulation task more difficult, as the adversary must modify both the totem and image pixels in a geometrically consistent manner without knowing the physical properties of the totem. Unlike prior learning-based approaches, our method does not require training on datasets of specific manipulations, and instead uses physical properties of the scene and camera to solve the forensics problem.