SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine

TL;DR

SYNAPSE introduces a neuro-symbolic framework that efficiently learns user preferences from limited data, enabling better alignment of robot behaviors with individual subjective preferences through program synthesis and visual parsing.

Contribution

The paper presents a novel neuro-symbolic approach combining visual parsing, language models, and program synthesis to learn preferences from minimal data, outperforming baselines especially out-of-distribution.

Findings

Significantly outperforms baseline methods in preference learning tasks.

Demonstrates strong out-of-distribution generalization capabilities.

Provides insights into the importance of design choices via ablation studies.

Abstract

This paper addresses the problem of preference learning, which aims to align robot behaviors through learning user specific preferences (e.g. "good pull-over location") from visual demonstrations. Despite its similarity to learning factual concepts (e.g. "red door"), preference learning is a fundamentally harder problem due to its subjective nature and the paucity of person-specific training data. We address this problem using a novel framework called SYNAPSE, which is a neuro-symbolic approach designed to efficiently learn preferential concepts from limited data. SYNAPSE represents preferences as neuro-symbolic programs, facilitating inspection of individual parts for alignment, in a domain-specific language (DSL) that operates over images and leverages a novel combination of visual parsing, large language models, and program synthesis to learn programs representing individual preferences. We perform extensive evaluations on various preferential concepts as well as user case studies demonstrating its ability to align well with dissimilar user preferences. Our method significantly outperforms baselines, especially when it comes to out of distribution generalization. We show the importance of the design choices in the framework through multiple ablation studies. Code, additional results, and supplementary material can be found on the website: https://amrl.cs.utexas.edu/synapse