Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report

TL;DR

This paper reports on a challenge to develop real-time, quantized image super-resolution solutions optimized for mobile NPUs, achieving high-quality 3X upscaling on constrained hardware with efficient inference times.

Contribution

It introduces the first Mobile AI challenge focused on real-time, quantized super-resolution on mobile devices, providing datasets, benchmarks, and detailed model descriptions.

Findings

Models can upscale images to Full HD in under 60 ms.

Proposed solutions are compatible with major mobile AI accelerators.

High fidelity results achieved with quantized models on constrained hardware.

Abstract

Image super-resolution is one of the most popular computer vision problems with many important applications to mobile devices. While many solutions have been proposed for this task, they are usually not optimized even for common smartphone AI hardware, not to mention more constrained smart TV platforms that are often supporting INT8 inference only. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a real-time performance on mobile or edge NPUs. For this, the participants were provided with the DIV2K dataset and trained quantized models to do an efficient 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated NPU capable of accelerating quantized neural networks. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms while achieving high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.