Weakly Supervised Correspondence Learning

TL;DR

This paper introduces a weakly supervised approach for correspondence learning in robotics, combining temporal ordering and paired abstractions to improve accuracy while reducing the need for costly strict pairings.

Contribution

It proposes a novel weak supervision framework that balances between strict pairing and unsupervised methods, utilizing easily accessible real-world signals.

Findings

Reduces misalignment issues compared to unsupervised methods

Leverages weak supervision to improve correspondence accuracy

Eases data annotation requirements in real-world applications

Abstract

Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments. However, current correspondence learning methods either leverage strictly paired data -- which are often difficult to collect -- or learn in an unsupervised fashion from unpaired data using regularization techniques such as cycle-consistency -- which suffer from severe misalignment issues. We propose a weakly supervised correspondence learning approach that trades off between strong supervision over strictly paired data and unsupervised learning with a regularizer over unpaired data. Our idea is to leverage two types of weak supervision: i) temporal ordering of states and actions to reduce the compounding error, and ii) paired abstractions, instead of paired data, to alleviate the misalignment problem and learn a more accurate correspondence. The two types of weak supervision are easy to access in real-world applications, which simultaneously reduces the high cost of annotating strictly paired data and improves the quality of the learned correspondence.