Learning to map between ferns with differentiable binary embedding networks

TL;DR

This paper introduces a differentiable binary fern layer as a multiplication-free alternative to convolutional layers in deep networks, achieving competitive results with reduced computational complexity.

Contribution

It presents a novel differentiable binary fern layer that can replace convolutional layers, enabling more efficient deep learning models.

Findings

Outperforms binary XNOR net on TUPAC'16 classification

Achieves near state-of-the-art accuracy with fewer parameters

Demonstrates the viability of differentiable binary ferns in end-to-end training

Abstract

Current deep learning methods are based on the repeated, expensive application of convolutions with parameter-intensive weight matrices. In this work, we present a novel concept that enables the application of differentiable random ferns in end-to-end networks. It can then be used as multiplication-free convolutional layer alternative in deep network architectures. Our experiments on the binary classification task of the TUPAC'16 challenge demonstrate improved results over the state-of-the-art binary XNOR net and only slightly worse performance than its 2x more parameter intensive floating point CNN counterpart.