Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
Andrey Ignatov, Radu Timofte, Shuai Liu, Chaoyu Feng and, Furui Bai, Xiaotao Wang, Lei Lei, Ziyao Yi, Yan Xiang, Zibin, Liu, Shaoqing Li, Keming Shi, Dehui Kong, Ke Xu, Minsu Kwon, and Yaqi Wu, Jiesi Zheng, Zhihao Fan, Xun Wu, Feng Zhang and, Albert No, Minhyeok Cho, Zewen Chen

TL;DR
This paper reports on the development of efficient deep learning-based image signal processing pipelines for mobile GPUs, capable of real-time high-fidelity image enhancement on smartphones, based on the AIM 2022 challenge.
Contribution
It introduces novel end-to-end AI models optimized for mobile GPU deployment, demonstrating real-time processing of high-quality images using TensorFlow Lite.
Findings
Models process Full HD images in under 50 ms
Achieved high fidelity image quality comparable to professional cameras
Solutions are compatible with recent mobile GPUs
Abstract
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Enhancement Techniques · Advanced Image and Video Retrieval Techniques · Advanced Neural Network Applications
