Customizing an LLM for Enterprise Software Engineering

TL;DR

This paper presents Gemini for Google, a specialized large language model adapted for enterprise software engineering, demonstrating significant improvements in developer productivity through extensive data curation and tailored training strategies.

Contribution

It introduces a comprehensive methodology for customizing LLMs for enterprise use, including data extraction, preparation, and deployment, with empirical validation on Google's internal ecosystem.

Findings

Gemini for Google reduced iterations per turn by 23%

Increased code survival rates by approximately 17%

Provided a blueprint for enterprise model adaptation

Abstract

Enterprise software development is a continuous evolutionary process, characterized by incremental additions, architectural revisions, production deployments and rigorous maintenance. These activities generate valuable data that modern LLMs could be finetuned on, to unlock additional tool possibilities for enterprise software engineering. While frontier LLMs are already very capable, this form of customization offers a compelling path for enterprise-specific optimization. We introduce Gemini for Google (GfG)}, an adaptation of Gemini specialized for Google's internal software engineering ecosystem. This paper details the model's end-to-end development, from curating a trillion-token proprietary dataset to implementing a mid-training strategy that mitigates catastrophic forgetting. In a large-scale blind A/B study across 29,000 developers, Gemini for Google significantly outperformed baselines: reducing the mean number of iterations per turn by 23\%, and increasing code survival rates by about 17%. Beyond metrics, we provide a comprehensive blueprint for enterprise model adaptation, covering: (1)The extraction of high-value signals from software engineering data, (2)Data preparation strategies, (3)Full-stack model tuning (continued pre-training and post-training), and (4)The deployment of downstream applications. We believe this methodology offers a replicable path for other organizations to unlock the full potential of their internal engineering data.