Ensemble Mask Networks

TL;DR

This paper explores how feedforward neural networks can be designed with masking and pruning mechanisms to learn matrix-vector multiplication, enabling analysis of dependencies and interactions in graph models.

Contribution

It introduces novel masking and pruning techniques that allow neural networks to learn and represent matrix operations and dependency structures.

Findings

Networks can approximate matrix-vector multiplication

Masking and pruning reveal dependency structures

Applications to graph interaction analysis

Abstract

Can an $\mathbb{R}^n\rightarrow \mathbb{R}^n$ feedforward network learn matrix-vector multiplication? This study introduces two mechanisms - flexible masking to take matrix inputs, and a unique network pruning to respect the mask's dependency structure. Networks can approximate fixed operations such as matrix-vector multiplication $\phi(A,x) \rightarrow Ax$, motivating the mechanisms introduced with applications towards litmus-testing dependencies or interaction order in graph-based models.