Towards a Small Language Model Lifecycle Framework

TL;DR

This paper proposes a comprehensive, modular lifecycle framework for Small Language Models (SLMs) that integrates insights from literature and practice to improve development, deployment, and maintenance.

Contribution

It introduces a unified, structured lifecycle model for SLMs, emphasizing modularity, interconnections, and lifecycle-awareness to guide future research and practical applications.

Findings

A modular lifecycle model for SLMs is proposed.

The framework supports method reuse and co-adaptation.

It bridges theory and practice for SLM development.

Abstract

Background: The growing demand for efficient and deployable language models has led to increased interest in Small Language Models (SLMs). However, existing research remains fragmented, lacking a unified lifecycle perspective. Objective: This study aims to define a comprehensive lifecycle framework for SLMs by synthesizing insights from academic literature and practitioner sources. Method: We conducted a comprehensive survey of 36 works, analyzing and categorizing lifecycle-relevant techniques. Results: We propose a modular lifecycle model structured into main, optional, and cross-cutting components. The model captures key interconnections across stages, supporting method reuse, co-adaptation, and lifecycle-awareness. Conclusion: Our framework provides a coherent foundation for developing and maintaining SLMs, bridging theory and practice, and guiding future research and tool development.