A Credit Assignment Compiler for Joint Prediction

TL;DR

This paper introduces a credit assignment compiler that simplifies programming and improves efficiency in joint prediction tasks involving multiple dependent variables, demonstrating high accuracy with reduced complexity.

Contribution

It presents a novel compiler that defines search spaces via imperative programs, significantly reducing programming complexity and runtime for joint prediction models.

Findings

Achieves accuracy comparable to existing methods

Reduces programming and execution time substantially

Demonstrates effectiveness on multiple joint prediction tasks

Abstract

Many machine learning applications involve jointly predicting multiple mutually dependent output variables. Learning to search is a family of methods where the complex decision problem is cast into a sequence of decisions via a search space. Although these methods have shown promise both in theory and in practice, implementing them has been burdensomely awkward. In this paper, we show the search space can be defined by an arbitrary imperative program, turning learning to search into a credit assignment compiler. Altogether with the algorithmic improvements for the compiler, we radically reduce the complexity of programming and the running time. We demonstrate the feasibility of our approach on multiple joint prediction tasks. In all cases, we obtain accuracies as high as alternative approaches, at drastically reduced execution and programming time.