Embedding Novel Views in a Single JPEG Image

TL;DR

This paper introduces a method to embed multiple novel views within a single JPEG image by encoding MPI layers, enabling high-fidelity view synthesis while maintaining perceptual quality and robustness to common image modifications.

Contribution

It presents a novel technique to embed 3D view synthesis data into JPEG images, allowing for high-quality view restoration and robustness to compression and editing.

Findings

Restores high-fidelity novel views from JPEG images

Robust to JPEG compression, color adjustments, and cropping

Achieves 1024 bits per pixel embedding capacity

Abstract

We propose a novel approach for embedding novel views in a single JPEG image while preserving the perceptual fidelity of the modified JPEG image and the restored novel views. We adopt the popular novel view synthesis representation of multiplane images (MPIs). Our model first encodes 32 MPI layers (totally 128 channels) into a 3-channel JPEG image that can be decoded for MPIs to render novel views, with an embedding capacity of 1024 bits per pixel. We conducted experiments on public datasets with different novel view synthesis methods, and the results show that the proposed method can restore high-fidelity novel views from a slightly modified JPEG image. Furthermore, our method is robust to JPEG compression, color adjusting, and cropping. Our source code will be publicly available.