Pruning deep neural networks generates a sparse, bio-inspired nonlinear controller for insect flight

TL;DR

This paper presents a bio-inspired approach combining neural network pruning and model predictive control to efficiently model and control insect flight dynamics, revealing limits of sparsification for maintaining performance.

Contribution

It introduces a novel framework that uses network pruning inspired by natural systems to create sparse neural controllers for insect flight, with quantified limits on sparsification.

Findings

Networks can be pruned to about 7% of original weights while retaining control capabilities.

Performance drops sharply below a critical sparsity threshold.

Statistical analysis of weight distributions during pruning is provided.

Abstract

Insect flight is a strongly nonlinear and actuated dynamical system. As such, strategies for understanding its control have typically relied on either model-based methods or linearizations thereof. Here we develop a framework that combines model predictive control on an established flight dynamics model and deep neural networks (DNN) to create an efficient method for solving the inverse problem of flight control. We turn to natural systems for inspiration since they inherently demonstrate network pruning with the consequence of yielding more efficient networks for a specific set of tasks. This bio-inspired approach allows us to leverage network pruning to optimally sparsify a DNN architecture in order to perform flight tasks with as few neural connections as possible, however, there are limits to sparsification. Specifically, as the number of connections falls below a critical threshold, flight performance drops considerably. We develop sparsification paradigms and explore their limits for control tasks. Monte Carlo simulations also quantify the statistical distribution of network weights during pruning given initial random weights of the DNNs. We demonstrate that on average, the network can be pruned to retain approximately 7% of the original network weights, with statistical distributions quantified at each layer of the network. Overall, this work shows that sparsely connected DNNs are capable of predicting the forces required to follow flight trajectories. Additionally, sparsification has sharp performance limits.