Optimal Radiometric Calibration for Camera-Display Communication

TL;DR

This paper introduces a novel camera-display communication method that embeds and recovers hidden messages through radiometric calibration and photometric modeling, enabling robust message decoding in real-world scenes.

Contribution

The paper presents an innovative algorithm that performs simultaneous radiometric calibration and message recovery using a convex optimization framework based on a camera-display transfer function.

Findings

Effective communication between cameras and displays using embedded messages.

Robust message recovery across nine commercial camera-display pairs.

Efficient online radiometric calibration improves decoding accuracy.

Abstract

We present a novel method for communicating between a camera and display by embedding and recovering hidden and dynamic information within a displayed image. A handheld camera pointed at the display can receive not only the display image, but also the underlying message. These active scenes are fundamentally different from traditional passive scenes like QR codes because image formation is based on display emittance, not surface reflectance. Detecting and decoding the message requires careful photometric modeling for computational message recovery. Unlike standard watermarking and steganography methods that lie outside the domain of computer vision, our message recovery algorithm uses illumination to optically communicate hidden messages in real world scenes. The key innovation of our approach is an algorithm that performs simultaneous radiometric calibration and message recovery in one convex optimization problem. By modeling the photometry of the system using a camera-display transfer function (CDTF), we derive a physics-based kernel function for support vector machine classification. We demonstrate that our method of optimal online radiometric calibration (OORC) leads to an efficient and robust algorithm for computational messaging between nine commercial cameras and displays.