Eliminating Catastrophic Interference with Biased Competition
Amelia Elizabeth Pollard, Jonathan L. Shapiro

TL;DR
This paper introduces a neural network model inspired by neuroscience to improve multi-task learning by separating tasks and avoiding catastrophic interference, demonstrated on visual question answering datasets.
Contribution
The paper proposes a novel end-to-end multi-task learning model based on biased competition, eliminating the need for task labels and reformatting, inspired by neuronal attention theory.
Findings
Eliminates catastrophic interference in multi-task learning.
Achieves competitive results on visual question answering datasets.
Introduces a new toy dataset, MNIST-QA, for low-dimensional VQA testing.
Abstract
We present here a model to take advantage of the multi-task nature of complex datasets by learning to separate tasks and subtasks in and end to end manner by biasing competitive interactions in the network. This method does not require additional labelling or reformatting of data in a dataset. We propose an alternate view to the monolithic one-task-fits-all learning of multi-task problems, and describe a model based on a theory of neuronal attention from neuroscience, proposed by Desimone. We create and exhibit a new toy dataset, based on the MNIST dataset, which we call MNIST-QA, for testing Visual Question Answering architectures in a low-dimensional environment while preserving the more difficult components of the Visual Question Answering task, and demonstrate the proposed network architecture on this new dataset, as well as on COCO-QA and DAQUAR-FULL. We then demonstrate that this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Optimization and Search Problems · Machine Learning and Algorithms
