Deep Octonion Networks

TL;DR

This paper introduces deep octonion networks (DONs), extending complex and quaternion neural networks, demonstrating improved convergence and accuracy in image classification tasks on CIFAR datasets.

Contribution

The paper develops a general framework for DONs, including key components like octonion convolution and normalization, and shows their superior performance over existing deep networks.

Findings

DONs outperform DRNs, DCNs, and DQNs in accuracy

DONs exhibit better convergence properties

DONs are effective for image classification on CIFAR datasets

Abstract

Deep learning is a research hot topic in the field of machine learning. Real-value neural networks (Real NNs), especially deep real networks (DRNs), have been widely used in many research fields. In recent years, the deep complex networks (DCNs) and the deep quaternion networks (DQNs) have attracted more and more attentions. The octonion algebra, which is an extension of complex algebra and quaternion algebra, can provide more efficient and compact expression. This paper constructs a general framework of deep octonion networks (DONs) and provides the main building blocks of DONs such as octonion convolution, octonion batch normalization and octonion weight initialization; DONs are then used in image classification tasks for CIFAR-10 and CIFAR-100 data sets. Compared with the DRNs, the DCNs, and the DQNs, the proposed DONs have better convergence and higher classification accuracy. The success of DONs is also explained by multi-task learning.