BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons

TL;DR

This paper introduces BiMLP, a novel binary architecture for vision MLPs that enhances representation capacity through a multi-branch binary block, achieving state-of-the-art accuracy on ImageNet-1k.

Contribution

The paper proposes a new binary block with multiple branches and shortcut connections to improve binary MLP performance, addressing previous limitations in representation capacity.

Findings

BiMLP achieves state-of-the-art accuracy among binary CNNs on ImageNet-1k.

The proposed binary block effectively enriches the representation ability of binary FC layers.

Downsampling design reduces computational complexity while maintaining performance.

Abstract

This paper studies the problem of designing compact binary architectures for vision multi-layer perceptrons (MLPs). We provide extensive analysis on the difficulty of binarizing vision MLPs and find that previous binarization methods perform poorly due to limited capacity of binary MLPs. In contrast with the traditional CNNs that utilizing convolutional operations with large kernel size, fully-connected (FC) layers in MLPs can be treated as convolutional layers with kernel size $1\times1$. Thus, the representation ability of the FC layers will be limited when being binarized, and places restrictions on the capability of spatial mixing and channel mixing on the intermediate features. To this end, we propose to improve the performance of binary MLP (BiMLP) model by enriching the representation ability of binary FC layers. We design a novel binary block that contains multiple branches to merge a series of outputs from the same stage, and also a universal shortcut connection that encourages the information flow from the previous stage. The downsampling layers are also carefully designed to reduce the computational complexity while maintaining the classification performance. Experimental results on benchmark dataset ImageNet-1k demonstrate the effectiveness of the proposed BiMLP models, which achieve state-of-the-art accuracy compared to prior binary CNNs. The MindSpore code is available at \url{https://gitee.com/mindspore/models/tree/master/research/cv/BiMLP}.