A Logic-Driven Framework for Consistency of Neural Models

TL;DR

This paper introduces a logic-based framework to improve the consistency of neural models by enforcing logical invariants, applicable to both labeled and unlabeled data, enhancing prediction reliability.

Contribution

It presents a novel, model-agnostic learning framework that regularizes neural models with logic rules to reduce internal inconsistency, applicable to natural language inference.

Findings

Enforcing logic invariants improves model consistency.

Framework works with both labeled and unlabeled data.

Enhances accuracy and consistency in natural language inference.

Abstract

While neural models show remarkable accuracy on individual predictions, their internal beliefs can be inconsistent across examples. In this paper, we formalize such inconsistency as a generalization of prediction error. We propose a learning framework for constraining models using logic rules to regularize them away from inconsistency. Our framework can leverage both labeled and unlabeled examples and is directly compatible with off-the-shelf learning schemes without model redesign. We instantiate our framework on natural language inference, where experiments show that enforcing invariants stated in logic can help make the predictions of neural models both accurate and consistent.