Neuro-Symbolic Entropy Regularization
Kareem Ahmed, Eric Wang, Kai-Wei Chang, Guy Van den Broeck

TL;DR
This paper introduces a neuro-symbolic entropy regularization framework that improves structured prediction by encouraging models to confidently predict valid structures, leveraging both entropy regularization and symbolic constraints.
Contribution
It unifies entropy regularization with neuro-symbolic constraints, enabling efficient training of models that produce valid structured outputs with less labeled data.
Findings
Models achieve higher accuracy on structured prediction tasks.
Predictions are more valid and consistent with constraints.
Effective in semi-supervised and fully-supervised settings.
Abstract
In structured prediction, the goal is to jointly predict many output variables that together encode a structured object -- a path in a graph, an entity-relation triple, or an ordering of objects. Such a large output space makes learning hard and requires vast amounts of labeled data. Different approaches leverage alternate sources of supervision. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. It extracts supervision from unlabeled examples, but remains agnostic to the structure of the output space. Conversely, neuro-symbolic approaches exploit the knowledge that not every prediction corresponds to a valid structure in the output space. Yet, they does not further restrict the learned output distribution. This paper introduces a framework that unifies both approaches. We propose a loss, neuro-symbolic entropy…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI) · Machine Learning and Data Classification
MethodsEntropy Regularization
