Position: Avoid Overstretching LLMs for every Enterprise Task

TL;DR

This paper advocates for a modular approach to enterprise AI, emphasizing external knowledge bases and symbolic procedures over monolithic LLMs for better reliability, scalability, and transparency.

Contribution

It introduces a theoretical framework and position advocating for modular architectures that externalize knowledge and computation, challenging the reliance on large language models for all enterprise tasks.

Findings

Finite-capacity models cannot fully capture enterprise knowledge.

Modular architectures improve reliability and maintainability.

Externalizing knowledge enhances scalability and transparency.

Abstract

Enterprise workloads are dominated by deterministic, structured, and knowledge-dependent tasks operating under strict cost, latency, and reliability constraints. While these are often addressed through large language model (LLM) deployment or distillation into smaller models, we argue this is inefficient, unreliable, and misaligned with enterprise task structures. Instead, AI systems should treat language models as interfaces rather than monolithic engines, externalizing knowledge and computation into dedicated components for greater reliability, scalability, and transparency. Our theoretical evidences show that finite-capacity models cannot fully capture the breadth of knowledge required for enterprise tasks, creating inherent limits to efficiency and interpretability. Building on this, we take the position that language models should primarily be used for structured extraction in deterministic enterprise workflows, while computation and storage are delegated to knowledge bases and symbolic procedures. We formally demonstrate that such modular architectures are more reliable and maintainable than monolithic frameworks, offering a sustainable foundation for enterprise tasks.