# Beyond the Surface: Probing the Ideological Depth of Large Language Models

## TL;DR

This paper introduces the concept of ideological depth in large language models, measuring their political steerability and internal feature richness, revealing significant differences among models and implications for safety and capability assessments.

## Contribution

It defines and quantifies ideological depth, comparing models with new probing techniques, and demonstrates how internal political features influence model behavior.

## Key findings

- Gemma-2-9B-IT is more steerable than Llama-3.1-8B-Instruct.
- Gemma activates 7.3 times more political features.
- Targeted ablations of political features affect model responses.

## Abstract

Large language models (LLMs) display recognizable political leanings, yet they vary significantly in their ability to represent a political orientation consistently. In this paper, we define ideological depth as (i) a model's ability to follow political instructions without failure (steerability), and (ii) the feature richness of its internal political representations measured with sparse autoencoders (SAEs), an unsupervised sparse dictionary learning (SDL) approach. Using Llama-3.1-8B-Instruct and Gemma-2-9B-IT as candidates, we compare prompt-based and activation-steering interventions and probe political features with publicly available SAEs. We find large, systematic differences: Gemma is more steerable in both directions and activates approximately 7.3x more distinct political features than Llama. Furthermore, causal ablations of a small targeted set of Gemma's political features to create a similar feature-poor setting induce consistent shifts in its behavior, with increased rates of refusals across topics. Together, these results indicate that refusals on benign political instructions or prompts can arise from capability deficits rather than safety guardrails. Ideological depth thus emerges as a measurable property of LLMs, and steerability serves as a window into their latent political architecture.