Models Got Talent: Identifying High Performing Wearable Human Activity Recognition Models Without Training

TL;DR

This paper explores the use of Zero Cost Proxies (ZCPs) to identify high-performing human activity recognition models from wearable sensors without extensive training, saving computational resources.

Contribution

It demonstrates that ZCPs can effectively select near-optimal HAR architectures across multiple datasets with minimal training, introducing a practical approach for model selection.

Findings

ZCPs identify architectures within 5% of fully trained models.

ZCPs are robust to data noise in HAR tasks.

Significant computational savings in model discovery.

Abstract

A promising alternative to the computationally expensive Neural Architecture Search (NAS) involves the development of Zero Cost Proxies (ZCPs), which correlate well with trained performance, but can be computed through a single forward/backward pass on a randomly sampled batch of data. In this paper, we investigate the effectiveness of ZCPs for HAR on six benchmark datasets, and demonstrate that they discover network architectures that obtain within 5% of performance attained by full-scale training involving 1500 randomly sampled architectures. This results in substantial computational savings as high-performing architectures can be discovered with minimal training. Our experiments not only introduce ZCPs to sensor-based HAR, but also demonstrate that they are robust to data noise, further showcasing their suitability for practical scenarios.