Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance

TL;DR

This paper introduces a neural network editing framework using a graph metanetwork trained to efficiently modify models for compliance with various requirements, balancing utility and requirement satisfaction.

Contribution

It presents a novel, data-driven approach to model editing with a metanetwork that enables single-step adjustments without retraining or performance loss.

Findings

Improves trade-offs between performance and requirement satisfaction.

Reduces time compared to post-processing or retraining.

Effective across tasks like bias mitigation and weight pruning.

Abstract

As machine learning models are increasingly deployed in high-stakes settings, e.g. as decision support systems in various societal sectors or in critical infrastructure, designers and auditors are facing the need to ensure that models satisfy a wider variety of requirements (e.g. compliance with regulations, fairness, computational constraints) beyond performance. Although most of them are the subject of ongoing studies, typical approaches face critical challenges: post-processing methods tend to compromise performance, which is often counteracted by fine-tuning or, worse, training from scratch, an often time-consuming or even unavailable strategy. This raises the following question: "Can we efficiently edit models to satisfy requirements, without sacrificing their utility?" In this work, we approach this with a unifying framework, in a data-driven manner, i.e. we learn to edit neural networks (NNs), where the editor is an NN itself - a graph metanetwork - and editing amounts to a single inference step. In particular, the metanetwork is trained on NN populations to minimise an objective consisting of two terms: the requirement to be enforced and the preservation of the NN's utility. We experiment with diverse tasks (the data minimisation principle, bias mitigation and weight pruning) improving the trade-offs between performance, requirement satisfaction and time efficiency compared to popular post-processing or re-training alternatives.