DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching

TL;DR

This paper introduces DGA-Net, a novel attention model that combines static and dynamic attention mechanisms to improve sentence semantic matching by capturing both global and local contextual information.

Contribution

The paper proposes a Dynamic Gaussian Attention (DGA) mechanism integrated into a new network, enhancing attention effectiveness in sentence semantic matching tasks.

Findings

DGA-Net outperforms existing models on benchmark datasets.

The model effectively captures both global and local semantic information.

Experimental results show significant accuracy improvements.

Abstract

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, where much recent progress has been made by the advancement of representation learning techniques and inspiration of human behaviors. Among all these methods, attention mechanism plays an essential role by selecting important parts effectively. However, current attention methods either focus on all the important parts in a static way or only select one important part at one attention step dynamically, which leaves a large space for further improvement. To this end, in this paper, we design a novel Dynamic Gaussian Attention Network (DGA-Net) to combine the advantages of current static and dynamic attention methods. More specifically, we first leverage pre-trained language model to encode the input sentences and construct semantic representations from a global perspective. Then, we develop a Dynamic Gaussian Attention (DGA) to dynamically capture the important parts and corresponding local contexts from a detailed perspective. Finally, we combine the global information and detailed local information together to decide the semantic relation of sentences comprehensively and precisely. Extensive experiments on two popular sentence semantic matching tasks demonstrate that our proposed DGA-Net is effective in improving the ability of attention mechanism.