# Attention Strategies for Multi-Source Sequence-to-Sequence Learning

## TL;DR

This paper introduces two new attention combination methods for multi-source sequence-to-sequence models, demonstrating their effectiveness on translation and post-editing tasks with competitive results.

## Contribution

It proposes flat and hierarchical attention combination strategies specifically designed for multi-source sequence-to-sequence learning, filling a gap in existing attention modeling research.

## Key findings

- Both methods achieve competitive results on WMT16 Multimodal Translation.
- Hierarchical approach outperforms some existing techniques.
- Systematic evaluation confirms effectiveness of proposed methods.

## Abstract

Modeling attention in neural multi-source sequence-to-sequence learning remains a relatively unexplored area, despite its usefulness in tasks that incorporate multiple source languages or modalities. We propose two novel approaches to combine the outputs of attention mechanisms over each source sequence, flat and hierarchical. We compare the proposed methods with existing techniques and present results of systematic evaluation of those methods on the WMT16 Multimodal Translation and Automatic Post-editing tasks. We show that the proposed methods achieve competitive results on both tasks.