Tracing Pharmacological Knowledge In Large Language Models

TL;DR

This paper investigates how pharmacological knowledge, specifically drug-group semantics, is represented within Llama-based biomedical language models, revealing that such knowledge is distributed across tokens and layers rather than localized.

Contribution

It provides the first systematic mechanistic analysis of pharmacological knowledge encoding in large language models using interpretability methods.

Findings

Early layers encode drug-group knowledge.

Distributed representations of pharmacological semantics are present in embeddings.

Intermediate tokens have a stronger causal effect on encoding drug-group information.

Abstract

Large language models (LLMs) have shown strong empirical performance across pharmacology and drug discovery tasks, yet the internal mechanisms by which they encode pharmacological knowledge remain poorly understood. In this work, we investigate how drug-group semantics are represented and retrieved within Llama-based biomedical language models using causal and probing-based interpretability methods. We apply activation patching to localize where drug-group information is stored across model layers and token positions, and complement this analysis with linear probes trained on token-level and sum-pooled activations. Our results demonstrate that early layers play a key role in encoding drug-group knowledge, with the strongest causal effects arising from intermediate tokens within the drug-group span rather than the final drug-group token. Linear probing further reveals that pharmacological semantics are distributed across tokens and are already present in the embedding space, with token-level probes performing near chance while sum-pooled representations achieve maximal accuracy. Together, these findings suggest that drug-group semantics in LLMs are not localized to single tokens but instead arise from distributed representations. This study provides the first systematic mechanistic analysis of pharmacological knowledge in LLMs, offering insights into how biomedical semantics are encoded in large language models.