SLIM-LLMs: Modeling of Style-Sensory Language RelationshipsThrough Low-Dimensional Representations

TL;DR

This paper introduces SLIM-LLMs, a low-parameter model that captures style-sensory language relationships using low-dimensional LIWC features, achieving comparable performance to larger models across genres.

Contribution

It presents a novel R4 approach for low-dimensional stylistic representation and the SLIM-LLMs model that efficiently predicts sensorial language with fewer parameters.

Findings

Low-dimensional LIWC features (r=24) effectively capture stylistic information.

SLIM-LLMs match larger models' performance with up to 80% fewer parameters.

Model performs well across multiple genres.

Abstract

Sensorial language -- the language connected to our senses including vision, sound, touch, taste, smell, and interoception, plays a fundamental role in how we communicate experiences and perceptions. We explore the relationship between sensorial language and traditional stylistic features, like those measured by LIWC, using a novel Reduced-Rank Ridge Regression (R4) approach. We demonstrate that low-dimensional latent representations of LIWC features r = 24 effectively capture stylistic information for sensorial language prediction compared to the full feature set (r = 74). We introduce Stylometrically Lean Interpretable Models (SLIM-LLMs), which model non-linear relationships between these style dimensions. Evaluated across five genres, SLIM-LLMs with low-rank LIWC features match the performance of full-scale language models while reducing parameters by up to 80%.