Eliciting Instruction-tuned Code Language Models' Capabilities to Utilize Auxiliary Function for Code Generation

TL;DR

This paper investigates how instruction-tuned code language models can effectively utilize auxiliary functions to improve code generation, demonstrating methods to incorporate auxiliary functions and showing their effectiveness through experiments.

Contribution

It introduces methods for integrating auxiliary functions into instruction-tuned code models, enhancing their code generation capabilities beyond existing approaches.

Findings

Models using auxiliary functions outperform baseline models.

Proposed methods surpass recent proprietary models like GPT-4o.

Open-source models with auxiliary functions achieve state-of-the-art results.

Abstract

We study the code generation behavior of instruction-tuned models built on top of code pre-trained language models when they could access an auxiliary function to implement a function. We design several ways to provide auxiliary functions to the models by adding them to the query or providing a response prefix to incorporate the ability to utilize auxiliary functions with the instruction-following capability. Our experimental results show the effectiveness of combining the base models' auxiliary function utilization ability with the instruction following ability. In particular, the performance of adopting our approaches with the open-sourced language models surpasses that of the recent powerful proprietary language models, i.e., gpt-4o.