Parameter Sharing Decoder Pair for Auto Composing

TL;DR

This paper introduces a parameter sharing decoder pair (PSDP) that significantly reduces model size while maintaining high-quality text auto composing, leveraging transformer-based models.

Contribution

The paper proposes a novel parameter sharing decoder pair to decrease parameters in transformer models for auto composing tasks, enhancing efficiency without sacrificing performance.

Findings

Reduces model parameters dramatically

Maintains understandable and reasonable compositions

Demonstrates effectiveness through generated works

Abstract

Auto Composing is an active and appealing research area in the past few years, and lots of efforts have been put into inventing more robust models to solve this problem. With the fast evolution of deep learning techniques, some deep neural network-based language models are becoming dominant. Notably, the transformer structure has been proven to be very efficient and promising in modeling texts. However, the transformer-based language models usually contain huge number of parameters and the size of the model is usually too large to put in production for some storage limited applications. In this paper, we propose a parameter sharing decoder pair (PSDP), which reduces the number of parameters dramatically and at the same time maintains the capability of generating understandable and reasonable compositions. Works created by the proposed model are presented to demonstrate the effectiveness of the model.