Dopamine: Brain Modes, Not Brains
Shervin Ghasemlou

TL;DR
This paper introduces extsc{ModeTune}, a neuromodulation-inspired PEFT method that adapts models by learning neuron thresholds and gains, enabling interpretable mode switching with fewer parameters than traditional methods.
Contribution
extsc{ModeTune} offers a novel activation-space PEFT approach that freezes base weights and learns neuron thresholds and gains for interpretable mode adaptation.
Findings
Improves rotated MNIST accuracy with few trainable parameters
Enables explicit neuron-level attributions and conditional computation
Exhibits partial activation sparsity
Abstract
Parameter-efficient fine-tuning (PEFT) methods such as \lora{} adapt large pretrained models by adding small weight-space updates. While effective, weight deltas are hard to interpret mechanistically, and they do not directly expose \emph{which} internal computations are reused versus bypassed for a new task. We explore an alternative view inspired by neuromodulation: adaptation as a change in \emph{mode} -- selecting and rescaling existing computations -- rather than rewriting the underlying weights. We propose \methodname{}, a simple activation-space PEFT technique that freezes base weights and learns per-neuron \emph{thresholds} and \emph{gains}. During training, a smooth gate decides whether a neuron's activation participates; at inference the gate can be hardened to yield explicit conditional computation and neuron-level attributions. As a proof of concept, we study ``mode…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeurological disorders and treatments · Neural dynamics and brain function · Functional Brain Connectivity Studies
