Landscape of Neural Architecture Search across sensors: how much do they differ ?

TL;DR

This study applies the Fitness Landscape Footprint framework to analyze neural architecture search across different sensors in image classification, revealing similar landscape behaviors and identifying the most beneficial sensor for hyper-parameter optimization.

Contribution

It extends the Fitness Landscape Footprint methodology to sensor-based neural architecture search, providing insights into landscape similarities and sensor effectiveness in real-world image classification.

Findings

Longer training increases fitness and landscape flatness.

Sentinel-2 sensor yields better fitness and smoother search trajectories.

Sensors with higher fitness enable more effective search behaviors.

Abstract

With the rapid rise of neural architecture search, the ability to understand its complexity from the perspective of a search algorithm is desirable. Recently, Traor\'e et al. have proposed the framework of Fitness Landscape Footprint to help describe and compare neural architecture search problems. It attempts at describing why a search strategy might be successful, struggle or fail on a target task. Our study leverages this methodology in the context of searching across sensors, including sensor data fusion. In particular, we apply the Fitness Landscape Footprint to the real-world image classification problem of So2Sat LCZ42, in order to identify the most beneficial sensor to our neural network hyper-parameter optimization problem. From the perspective of distributions of fitness, our findings indicate a similar behaviour of the search space for all sensors: the longer the training time, the larger the overall fitness, and more flatness in the landscapes (less ruggedness and deviation). Regarding sensors, the better the fitness they enable (Sentinel-2), the better the search trajectories (smoother, higher persistence). Results also indicate very similar search behaviour for sensors that can be decently fitted by the search space (Sentinel-2 and fusion).