# Learning Longer-term Dependencies via Grouped Distributor Unit

## TL;DR

This paper introduces a novel gated RNN called Grouped Distributor Unit (GDU) that effectively captures long-term dependencies by partitioning hidden states into groups with adaptive update rates, outperforming LSTM and GRU.

## Contribution

The paper proposes a simpler gated RNN structure with grouped hidden states and adaptive memory updates, enhancing long-term dependency learning.

## Key findings

- GDU outperforms LSTM and GRU on various tasks.
- GDU has a simpler structure with only one gate.
- Flexible grouping helps in capturing long-term dependencies.

## Abstract

Learning long-term dependencies still remains difficult for recurrent neural networks (RNNs) despite their success in sequence modeling recently. In this paper, we propose a novel gated RNN structure, which contains only one gate. Hidden states in the proposed grouped distributor unit (GDU) are partitioned into groups. For each group, the proportion of memory to be overwritten in each state transition is limited to a constant and is adaptively distributed to each group member. In other word, every separate group has a fixed overall update rate, yet all units are allowed to have different paces. Information is therefore forced to be latched in a flexible way, which helps the model to capture long-term dependencies in data. Besides having a simpler structure, GDU is demonstrated experimentally to outperform LSTM and GRU on tasks including both pathological problems and natural data set.