Dynamic Network selection for the Object Detection task: why it matters and what we (didn't) achieve

TL;DR

This paper explores dynamic auto-tuning for object detection neural networks, demonstrating potential benefits in selecting optimal models based on quality and time constraints, but faces challenges in predictive feature identification.

Contribution

It introduces an adaptive methodology for switching networks during inference and develops an oracle to improve selection, highlighting the need for effective image features.

Findings

No single optimal network for COCO detection.

Time and quality metrics influence network selection.

Predictor functions for image features remain elusive.

Abstract

In this paper, we want to show the potential benefit of a dynamic auto-tuning approach for the inference process in the Deep Neural Network (DNN) context, tackling the object detection challenge. We benchmarked different neural networks to find the optimal detector for the well-known COCO 17 database, and we demonstrate that even if we only consider the quality of the prediction there is not a single optimal network. This is even more evident if we also consider the time to solution as a metric to evaluate, and then select, the most suitable network. This opens to the possibility for an adaptive methodology to switch among different object detection networks according to run-time requirements (e.g. maximum quality subject to a time-to-solution constraint). Moreover, we demonstrated by developing an ad hoc oracle, that an additional proactive methodology could provide even greater benefits, allowing us to select the best network among the available ones given some characteristics of the processed image. To exploit this method, we need to identify some image features that can be used to steer the decision on the most promising network. Despite the optimization opportunity that has been identified, we were not able to identify a predictor function that validates this attempt neither adopting classical image features nor by using a DNN classifier.