Software Engineering and Foundation Models: Insights from Industry Blogs Using a Jury of Foundation Models

TL;DR

This paper analyzes industry blogs to understand how foundation models are integrated into software engineering practices and how SE methodologies are applied to FMs, revealing practical trends and future directions.

Contribution

It provides the first practitioner's perspective on FM in SE by systematically analyzing blog posts, highlighting real-world applications and deployment strategies of foundation models.

Findings

Code generation is the most prominent FM4SE task.

FMs are used for code understanding, summarization, and API recommendation.

Most SE4FM posts focus on deployment, system architecture, and orchestration.

Abstract

Foundation models (FMs) such as large language models (LLMs) have significantly impacted many fields, including software engineering (SE). The interaction between SE and FMs has led to the integration of FMs into SE practices (FM4SE) and the application of SE methodologies to FMs (SE4FM). While several literature surveys exist on academic contributions to these trends, we are the first to provide a practitioner's view. We analyze 155 FM4SE and 997 SE4FM blog posts from leading technology companies, leveraging an FM-powered surveying approach to systematically label and summarize the discussed activities and tasks. We observed that while code generation is the most prominent FM4SE task, FMs are leveraged for many other SE activities such as code understanding, summarization, and API recommendation. The majority of blog posts on SE4FM are about model deployment & operation, and system architecture & orchestration. Although the emphasis is on cloud deployments, there is a growing interest in compressing FMs and deploying them on smaller devices such as edge or mobile devices. We outline eight future research directions inspired by our gained insights, aiming to bridge the gap between academic findings and real-world applications. Our study not only enriches the body of knowledge on practical applications of FM4SE and SE4FM but also demonstrates the utility of FMs as a powerful and efficient approach in conducting literature surveys within technical and grey literature domains. Our dataset, results, code and used prompts can be found in our online replication package at https://github.com/SAILResearch/fmse-blogs.