BanglaForge: LLM Collaboration with Self-Refinement for Bangla Code Generation

TL;DR

BanglaForge is a novel framework that combines retrieval, dual-model collaboration, and self-refinement to improve Bangla code generation from natural language descriptions, achieving high accuracy despite low-resource constraints.

Contribution

It introduces a new collaborative, self-refining approach for low-resource Bangla code generation, integrating retrieval, in-context learning, and iterative feedback.

Findings

Achieves 84.00% Pass@1 accuracy on BLP-2025 benchmark.

Demonstrates effectiveness of self-refinement and retrieval in low-resource settings.

Outperforms existing methods in Bangla code generation tasks.

Abstract

Bangla is a low-resource language for code generation, lacking large-scale annotated datasets and tools to transform natural language specifications into executable programs. This makes Bangla-to-code generation a challenging task requiring innovative solutions. To address this, we introduce BanglaForge, a novel framework for generating code from Bangla function descriptions. BanglaForge leverages a retrieval-augmented dual-model collaboration paradigm with self-refinement, combining in-context learning, llm-based translation, systematic prompt engineering, and iterative self-refinement based on execution feedback, where a coder generates initial solutions and a reviewer enhances them for robustness. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves a competitive Pass@1 accuracy of 84.00%, demonstrating the effectiveness of retrieval, model collaboration, and self-refinement for low-resource Bangla code generation.