A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion

TL;DR

This survey comprehensively reviews Dynamic Neural Networks in Computer Vision, emphasizing their adaptability, benefits in sensor fusion, and providing a curated repository of related research and code.

Contribution

It unifies existing research, introduces a taxonomy based on adaptive components, and highlights the potential of Dynamic Neural Networks in sensor fusion applications.

Findings

Dynamic Neural Networks improve computational efficiency for diverse inputs.

They are particularly beneficial for sensor fusion tasks.

A curated repository of related papers and code is provided.

Abstract

Model compression is essential in the deployment of large Computer Vision models on embedded devices. However, static optimization techniques (e.g. pruning, quantization, etc.) neglect the fact that different inputs have different complexities, thus requiring different amount of computations. Dynamic Neural Networks allow to condition the number of computations to the specific input. The current literature on the topic is very extensive and fragmented. We present a comprehensive survey that synthesizes and unifies existing Dynamic Neural Networks research in the context of Computer Vision. Additionally, we provide a logical taxonomy based on which component of the network is adaptive: the output, the computation graph or the input. Furthermore, we argue that Dynamic Neural Networks are particularly beneficial in the context of Sensor Fusion for better adaptivity, noise reduction and information prioritization. We present preliminary works in this direction. We complement this survey with a curated repository listing all the surveyed papers, each with a brief summary of the solution and the code base when available: https://github.com/DTU-PAS/awesome-dynn-for-cv .