SOTitle: A Transformer-based Post Title Generation Approach for Stack Overflow

TL;DR

This paper introduces SOTitle, a Transformer-based model that leverages code snippets and problem descriptions to automatically generate high-quality question titles on Stack Overflow, outperforming existing methods.

Contribution

The study proposes a novel multi-modal Transformer approach for post title generation that incorporates both code snippets and descriptions, and provides a large-scale high-quality dataset for evaluation.

Findings

SOTitle significantly outperforms six baseline models in automatic metrics.

SOTitle achieves higher human evaluation scores for title quality.

The approach effectively captures long-term dependencies in multi-modal data.

Abstract

On Stack Overflow, developers can not only browse question posts to solve their programming problems but also gain expertise from the question posts to help improve their programming skills. Therefore, improving the quality of question posts in Stack Overflow has attracted the wide attention of researchers. A concise and precise title can play an important role in helping developers understand the key information of the question post, which can improve the post quality. However, the quality of the generated title is not high due to the lack of professional knowledge related to their questions or the poor presentation ability of developers. A previous study aimed to automatically generate the title by analyzing the code snippets in the question post. However, this study ignored the useful information in the corresponding problem description. Therefore, we propose an approach SOTitle for automatic post title generation by leveraging the code snippets and the problem description in the question post (i.e., the multi-modal input). SOTitle follows the Transformer structure, which can effectively capture long-term dependencies through a multi-head attention mechanism. To verify the effectiveness of SOTitle, we construct a large-scale high-quality corpus from Stack Overflow, which includes 1,168,257 high-quality question posts for four popular programming languages. Experimental results show that SOTitle can significantly outperform six state-of-the-art baselines in both automatic evaluation and human evaluation. To encourage follow-up studies, we make our corpus and approach publicly available