Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations

TL;DR

The paper introduces Bridge Networks, a novel architecture combining information bottleneck theory and vector-symbolic architectures to address challenges like catastrophic forgetting, energy efficiency, and symbolic reasoning in deep learning.

Contribution

It proposes a new architecture that leverages vector-symbolic manipulations to improve global loss handling, reduce catastrophic forgetting, and enhance symbolic reasoning capabilities.

Findings

Addresses catastrophic forgetting effectively

Reduces energy consumption in deep learning models

Enables symbolic reasoning through vector manipulations

Abstract

Despite rapid progress, current deep learning methods face a number of critical challenges. These include high energy consumption, catastrophic forgetting, dependance on global losses, and an inability to reason symbolically. By combining concepts from information bottleneck theory and vector-symbolic architectures, we propose and implement a novel information processing architecture, the 'Bridge network.' We show this architecture provides unique advantages which can address the problem of global losses and catastrophic forgetting. Furthermore, we argue that it provides a further basis for increasing energy efficiency of execution and the ability to reason symbolically.