Temperature in SLMs: Impact on Incident Categorization in On-Premises Environments

TL;DR

This study evaluates the impact of temperature settings on locally executed SLMs for incident categorization, finding that temperature has minimal effect while model size and hardware are more influential.

Contribution

It provides an empirical analysis of how temperature hyperparameters affect the performance of SLMs in incident categorization tasks in on-premises environments.

Findings

Temperature has little influence on model performance.

Model size and GPU capacity are key factors affecting results.

Larger models and better hardware improve accuracy and efficiency.

Abstract

SOCs and CSIRTs face increasing pressure to automate incident categorization, yet the use of cloud-based LLMs introduces costs, latency, and confidentiality risks. We investigate whether locally executed SLMs can meet this challenge. We evaluated 21 models ranging from 1B to 20B parameters, varying the temperature hyperparameter and measuring execution time and precision across two distinct architectures. The results indicate that temperature has little influence on performance, whereas the number of parameters and GPU capacity are decisive factors.