The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning

TL;DR

The paper introduces gradiend, a Python package that facilitates gradient-based feature learning in language models, offering tools for data creation, training, evaluation, visualization, and model rewriting.

Contribution

It provides a comprehensive, open-source system for operationalizing the GRADIEND method in language models, enabling end-to-end feature learning workflows.

Findings

Demonstrates application on English pronouns

Reproduces prior feature comparison results

Provides a unified workflow for feature learning

Abstract

We present gradiend, an open-source Python package that operationalizes the GRADIEND method for learning feature directions from factual-counterfactual MLM and CLM gradients in language models. The package provides a unified workflow for feature-related data creation, training, evaluation, visualization, persistent model rewriting via controlled weight updates, and multi-feature comparison. We demonstrate GRADIEND on an English pronoun paradigm and on a large-scale feature comparison that reproduces prior use cases.