IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

TL;DR

IFOR is an end-to-end robotic object rearrangement method that uses learned optical flow and iterative minimization to accurately reposition unseen objects in cluttered scenes, trained solely on synthetic data.

Contribution

The paper introduces a novel approach combining learned optical flow with iterative minimization for unknown object rearrangement from RGBD images, effective in cluttered and real-world scenes.

Findings

Effective in cluttered scenes

Works with unseen objects

Trained solely on synthetic data

Abstract

Accurate object rearrangement from vision is a crucial problem for a wide variety of real-world robotics applications in unstructured environments. We propose IFOR, Iterative Flow Minimization for Robotic Object Rearrangement, an end-to-end method for the challenging problem of object rearrangement for unknown objects given an RGBD image of the original and final scenes. First, we learn an optical flow model based on RAFT to estimate the relative transformation of the objects purely from synthetic data. This flow is then used in an iterative minimization algorithm to achieve accurate positioning of previously unseen objects. Crucially, we show that our method applies to cluttered scenes, and in the real world, while training only on synthetic data. Videos are available at https://imankgoyal.github.io/ifor.html.