# Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering

## TL;DR

This paper investigates catastrophic forgetting in visual question answering models by designing linguistically-informed tasks, revealing that task difficulty and order significantly impact continual learning, with current methods only partially mitigating forgetting.

## Contribution

It introduces linguistically-informed VQA tasks inspired by psycholinguistics to study forgetting, highlighting the effects of task difficulty and order on continual learning.

## Key findings

- Significant catastrophic forgetting observed in VQA models.
- Task difficulty and question order influence forgetting.
- Existing continual learning methods only partially reduce forgetting.

## Abstract

We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.