Selective Neuron Amplification in Transformer Language Models

TL;DR

This paper introduces Selective Neuron Amplification, a method to enhance model performance by boosting relevant neuron activity during inference, addressing activation issues rather than knowledge gaps.

Contribution

The paper proposes a novel inference-time technique, SNA, that amplifies task-relevant neurons without altering model parameters, improving performance on uncertain tasks.

Findings

SNA increases model accuracy when uncertainty is high.

Weak neuron activation contributes to model failures.

SNA does not affect model confidence when already high.

Abstract

Large language models often fail on tasks they seem to already understand. In our experiments, this appears to be less about missing knowledge and more about certain internal circuits not being strongly activated during inference. We explore Selective Neuron Amplification, which increases the influence of task relevant neurons without changing the model's parameters. The method works at inference time and does not permanently alter the model. SNA helps mainly when the model is uncertain, while having low effect when the model is already confident. This suggests that some model failures are due to weak activation rather than lack of capability.