Metaphors We Learn By

TL;DR

This paper explores how parameter sharing in neural networks relates to analogy making and cognitive metaphors, suggesting a broader view of learning that unifies connectionist and rule-based approaches.

Contribution

It introduces a novel perspective linking parameter sharing to analogy and cognitive metaphor, extending the understanding of neural network learning.

Findings

Parameter sharing relates to analogy making in cognition.

Recurrent models extend analogy to dynamic skills.

Challenges the dichotomy between connectionist and rule-based computation.

Abstract

Gradient based learning using error back-propagation (``backprop'') is a well-known contributor to much of the recent progress in AI. A less obvious, but arguably equally important, ingredient is parameter sharing - most well-known in the context of convolutional networks. In this essay we relate parameter sharing (``weight sharing'') to analogy making and the school of thought of cognitive metaphor. We discuss how recurrent and auto-regressive models can be thought of as extending analogy making from static features to dynamic skills and procedures. We also discuss corollaries of this perspective, for example, how it can challenge the currently entrenched dichotomy between connectionist and ``classic'' rule-based views of computation.