SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device

TL;DR

SYENet is a lightweight, multi-task neural network designed for real-time low-level vision tasks on mobile devices, achieving high performance with only 6K parameters and suitable for applications like image processing and super-resolution.

Contribution

The paper introduces SYENet, a novel multi-task network with a simple architecture, a Quadratic Connection Unit, and an Outlier-Aware Loss, enabling efficient real-time low-level vision processing on mobile hardware.

Findings

Outperforms existing networks in PSNR for multiple tasks

Achieves 2K60FPS throughput on Qualcomm 8 Gen 1 SoC

Won the MAI 2022 Smartphone ISP challenge

Abstract

With the rapid development of AI hardware accelerators, applying deep learning-based algorithms to solve various low-level vision tasks on mobile devices has gradually become possible. However, two main problems still need to be solved: task-specific algorithms make it difficult to integrate them into a single neural network architecture, and large amounts of parameters make it difficult to achieve real-time inference. To tackle these problems, we propose a novel network, SYENet, with only $~$6K parameters, to handle multiple low-level vision tasks on mobile devices in a real-time manner. The SYENet consists of two asymmetrical branches with simple building blocks. To effectively connect the results by asymmetrical branches, a Quadratic Connection Unit(QCU) is proposed. Furthermore, to improve performance, a new Outlier-Aware Loss is proposed to process the image. The proposed method proves its superior performance with the best PSNR as compared with other networks in real-time applications such as Image Signal Processing(ISP), Low-Light Enhancement(LLE), and Super-Resolution(SR) with 2K60FPS throughput on Qualcomm 8 Gen 1 mobile SoC(System-on-Chip). Particularly, for ISP task, SYENet got the highest score in MAI 2022 Learned Smartphone ISP challenge.