Cross-Camera Deep Colorization

TL;DR

This paper introduces an end-to-end CNN for aligning and fusing images from dual-camera systems, enabling high-quality colorization across different camera types and resolutions, with significant performance improvements.

Contribution

The proposed method is the first to effectively align and fuse images from cross-domain dual-camera systems for colorization, demonstrating robustness and flexibility in practical scenarios.

Findings

Achieves around 10dB PSNR gain over state-of-the-art methods.

Effectively handles cross-domain and cross-scale image inputs.

Demonstrates robustness across various datasets and settings.

Abstract

In this paper, we consider the color-plus-mono dual-camera system and propose an end-to-end convolutional neural network to align and fuse images from it in an efficient and cost-effective way. Our method takes cross-domain and cross-scale images as input, and consequently synthesizes HR colorization results to facilitate the trade-off between spatial-temporal resolution and color depth in the single-camera imaging system. In contrast to the previous colorization methods, ours can adapt to color and monochrome cameras with distinctive spatial-temporal resolutions, rendering the flexibility and robustness in practical applications. The key ingredient of our method is a cross-camera alignment module that generates multi-scale correspondences for cross-domain image alignment. Through extensive experiments on various datasets and multiple settings, we validate the flexibility and effectiveness of our approach. Remarkably, our method consistently achieves substantial improvements, i.e., around 10dB PSNR gain, upon the state-of-the-art methods. Code is at: https://github.com/IndigoPurple/CCDC