Narrow Transformer: StarCoder-Based Java-LM For Desktop
Kamalkumar Rathinasamy, Balaji A J, Ankush Kumar, Gagan Gayari,, Harshini K, Rajab Ali Mondal, Sreenivasa Raghavan K S, Swayam Singh, Mohammed, Rafee Tarafdar

TL;DR
This paper introduces NT-Java-1.1B, a specialized Java code language model based on StarCoder, optimized for desktop deployment, achieving state-of-the-art performance among small models on Java benchmarks.
Contribution
Development of NT-Java-1.1B, a small, efficient Java code model that surpasses similar-sized models and is suitable for desktop use, filling a research gap in small language models for specific programming languages.
Findings
NT-Java-1.1B outperforms comparable models on MultiPL-E Java benchmark.
Quantized versions maintain high performance, enabling desktop deployment.
The model establishes a foundation for language-specific, size-optimized code models.
Abstract
This paper presents NT-Java-1.1B, an open-source specialized code language model built on StarCoderBase-1.1B, designed for coding tasks in Java programming. NT-Java-1.1B achieves state-of-the-art performance, surpassing its base model and majority of other models of similar size on MultiPL-E Java code benchmark. While there have been studies on extending large, generic pre-trained models to improve proficiency in specific programming languages like Python, similar investigations on small code models for other programming languages are lacking. Large code models require specialized hardware like GPUs for inference, highlighting the need for research into building small code models that can be deployed on developer desktops. This paper addresses this research gap by focusing on the development of a small Java code model, NT-Java-1.1B, and its quantized versions, which performs comparably…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModeling and Simulation Systems · Embedded Systems Design Techniques
MethodsBalanced Selection
