Survey of Specialized Large Language Model

TL;DR

This survey reviews the evolution of specialized large language models across various domains, highlighting technical innovations, performance improvements, and future implications for fields like E-Commerce.

Contribution

It provides a comprehensive overview of recent advancements in specialized LLM architectures, including domain-native designs and multimodal integration, and discusses their impact on professional applications.

Findings

Specialized LLMs outperform general models on domain-specific benchmarks.

Emergence of domain-native architectures beyond fine-tuning.

Increasing use of multimodal capabilities and parameter efficiency techniques.

Abstract

The rapid evolution of specialized large language models (LLMs) has transitioned from simple domain adaptation to sophisticated native architectures, marking a paradigm shift in AI development. This survey systematically examines this progression across healthcare, finance, legal, and technical domains. Besides the wide use of specialized LLMs, technical breakthrough such as the emergence of domain-native designs beyond fine-tuning, growing emphasis on parameter efficiency through sparse computation and quantization, increasing integration of multimodal capabilities and so on are applied to recent LLM agent. Our analysis reveals how these innovations address fundamental limitations of general-purpose LLMs in professional applications, with specialized models consistently performance gains on domain-specific benchmarks. The survey further highlights the implications for E-Commerce field to fill gaps in the field.