Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships

TL;DR

This paper introduces abstraction alignment, a method to compare model behavior with human knowledge by using an abstraction graph, helping to evaluate and improve model interpretability and alignment with human concepts.

Contribution

The paper presents a novel methodology called abstraction alignment that quantifies how well models learn human-aligned abstractions using a formal abstraction graph.

Findings

Differentiates similar errors based on human concepts

Enhances existing model-quality metrics with abstraction alignment

Identifies areas where human abstractions can be improved

Abstract

While interpretability methods identify a model's learned concepts, they overlook the relationships between concepts that make up its abstractions and inform its ability to generalize to new data. To assess whether models' have learned human-aligned abstractions, we introduce abstraction alignment, a methodology to compare model behavior against formal human knowledge. Abstraction alignment externalizes domain-specific human knowledge as an abstraction graph, a set of pertinent concepts spanning levels of abstraction. Using the abstraction graph as a ground truth, abstraction alignment measures the alignment of a model's behavior by determining how much of its uncertainty is accounted for by the human abstractions. By aggregating abstraction alignment across entire datasets, users can test alignment hypotheses, such as which human concepts the model has learned and where misalignments recur. In evaluations with experts, abstraction alignment differentiates seemingly similar errors, improves the verbosity of existing model-quality metrics, and uncovers improvements to current human abstractions.