MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning

TL;DR

MetaTPTrans introduces a meta learning method that dynamically generates language-specific parameters for multilingual code representation, effectively capturing both shared and unique features across programming languages, leading to improved software engineering task performance.

Contribution

The paper presents a novel meta learning approach that generates dynamic parameters for language-specific code features, addressing limitations of previous models that only learned shared representations.

Findings

Significant improvements on code summarization tasks.

Enhanced performance on code completion benchmarks.

Outperforms state-of-the-art baselines.

Abstract

Representation learning of source code is essential for applying machine learning to software engineering tasks. Learning code representation from a multilingual source code dataset has been shown to be more effective than learning from single-language datasets separately, since more training data from multilingual dataset improves the model's ability to extract language-agnostic information from source code. However, existing multilingual training overlooks the language-specific information which is crucial for modeling source code across different programming languages, while only focusing on learning a unified model with shared parameters among different languages for language-agnostic information modeling. To address this problem, we propose MetaTPTrans, a meta learning approach for multilingual code representation learning. MetaTPTrans generates different parameters for the feature extractor according to the specific programming language type of the input code snippet, enabling the model to learn both language-agnostic and language-specific information with dynamic parameters in the feature extractor. We conduct experiments on the code summarization and code completion tasks to verify the effectiveness of our approach. The results demonstrate the superiority of our approach with significant improvements on state-of-the-art baselines.