FastONN -- Python based open-source GPU implementation for Operational Neural Networks

TL;DR

FastONN is a GPU-accelerated Python library that efficiently trains Operational Neural Networks by introducing a vectorized formulation, supporting flexible operator sets, and providing tools for performance monitoring.

Contribution

It presents a novel, fast GPU implementation of ONNs with a vectorized approach, enabling flexible operator integration and enhanced training capabilities.

Findings

Significantly faster training times for ONNs on GPU

Supports diverse non-linear operators for neural network customization

Includes modules for performance tracking and checkpointing

Abstract

Operational Neural Networks (ONNs) have recently been proposed as a special class of artificial neural networks for grid structured data. They enable heterogenous non-linear operations to generalize the widely adopted convolution-based neuron model. This work introduces a fast GPU-enabled library for training operational neural networks, FastONN, which is based on a novel vectorized formulation of the operational neurons. Leveraging on automatic reverse-mode differentiation for backpropagation, FastONN enables increased flexibility with the incorporation of new operator sets and customized gradient flows. Additionally, bundled auxiliary modules offer interfaces for performance tracking and checkpointing across different data partitions and customized metrics.