# Zoom To Learn, Learn To Zoom

## TL;DR

This paper demonstrates that training on real RAW sensor data significantly improves digital zoom quality in photography, introducing a new dataset and a robust deep learning method for state-of-the-art results.

## Contribution

It introduces SR-RAW, a dataset of real sensor data for training, and a novel deep network with CoBi loss for improved digital zoom performance.

## Key findings

- State-of-the-art 4X and 8X zoom performance achieved
- Training on real sensor data outperforms RGB-based methods
- Synthesized data is less effective than real sensor data

## Abstract

This paper shows that when applying machine learning to digital zoom for photography, it is beneficial to use real, RAW sensor data for training. Existing learning-based super-resolution methods do not use real sensor data, instead operating on RGB images. In practice, these approaches result in loss of detail and accuracy in their digitally zoomed output when zooming in on distant image regions. We also show that synthesizing sensor data by resampling high-resolution RGB images is an oversimplified approximation of real sensor data and noise, resulting in worse image quality. The key barrier to using real sensor data for training is that ground truth high-resolution imagery is missing. We show how to obtain the ground-truth data with optically zoomed images and contribute a dataset, SR-RAW, for real-world computational zoom. We use SR-RAW to train a deep network with a novel contextual bilateral loss (CoBi) that delivers critical robustness to mild misalignment in input-output image pairs. The trained network achieves state-of-the-art performance in 4X and 8X computational zoom.