Themes of Building LLM-based Applications for Production: A Practitioner's View
Alina Mailach, Sebastian Simon, Johannes Dorn, Norbert Siegmund

TL;DR
This paper systematically maps practitioner discussions on deploying LLMs in production, highlighting key themes like design, architecture, capabilities, infrastructure, and ethical challenges to guide future development.
Contribution
It provides a comprehensive overview of practical considerations and challenges in deploying LLM-based applications, based on analysis of online practitioner discussions.
Findings
Retrieval-augmented generation (RAG) is a prevalent design focus.
Model capabilities and enhancement techniques are frequently discussed.
Infrastructure, tooling, and ethical challenges are key concerns.
Abstract
Background: Large language models (LLMs) have become a paramount interest of researchers and practitioners alike, yet a comprehensive overview of key considerations for those developing LLM-based systems is lacking. This study addresses this gap by collecting and mapping the topics practitioners discuss online, offering practical insights into where priorities lie in developing LLM-based applications. Method: We collected 189 videos from 2022 to 2024 from practitioners actively developing such systems and discussing various aspects they encounter during development and deployment of LLMs in production. We analyzed the transcripts using BERTopic, then manually sorted and merged the generated topics into themes, leading to a total of 20 topics in 8 themes. Results: The most prevalent topics fall within the theme Design & Architecture, with a strong focus on retrieval-augmented generation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsManufacturing Process and Optimization
