Application-oriented automatic hyperparameter optimization for spiking neural network prototyping

TL;DR

This paper presents an application-oriented hyperparameter optimization pipeline for spiking neural networks using the NNI toolkit, demonstrating its effectiveness through a use case and reviewing related works.

Contribution

It introduces a systematic HPO pipeline tailored for SNNs with the NNI toolkit and provides a practical example and literature review for SNN prototyping.

Findings

Effective HPO pipeline for SNNs demonstrated

Use case shows improved model performance

Review of related HPO approaches for SNNs

Abstract

Hyperparameter optimization (HPO) is of paramount importance in the development of high-performance, specialized artificial intelligence (AI) models, ranging from well-established machine learning (ML) solutions to the deep learning (DL) domain and the field of spiking neural networks (SNNs). The latter introduce further complexity due to the neuronal computational units and their additional hyperparameters, whose inadequate setting can dramatically impact the final model performance. At the cost of possible reduced generalization capabilities, the most suitable strategy to fully disclose the power of SNNs is to adopt an application-oriented approach and perform extensive HPO experiments. To facilitate these operations, automatic pipelines are fundamental, and their configuration is crucial. In this document, the Neural Network Intelligence (NNI) toolkit is used as reference framework to present one such solution, with a use case example providing evidence of the corresponding results. In addition, a summary of published works employing the presented pipeline is reported as a potential source of insights into application-oriented HPO experiments for SNN prototyping.