Coarse or Fine? Recognising Action End States without Labels
Davide Moltisanti, Hakan Bilen, Laura Sevilla-Lara, Frank Keller

TL;DR
This paper introduces a synthetic data augmentation method for recognizing the coarseness of cuts in images, enabling effective training of a model that generalizes well to real-world data without requiring labeled datasets.
Contribution
It proposes a novel augmentation technique to generate training data for end state recognition of cuts, applicable across different objects without needing object labels.
Findings
Model accurately recognizes cut coarseness in real images.
Synthetic data enables training with limited initial images.
Model generalizes to unseen objects despite domain gap.
Abstract
We focus on the problem of recognising the end state of an action in an image, which is critical for understanding what action is performed and in which manner. We study this focusing on the task of predicting the coarseness of a cut, i.e., deciding whether an object was cut "coarsely" or "finely". No dataset with these annotated end states is available, so we propose an augmentation method to synthesise training data. We apply this method to cutting actions extracted from an existing action recognition dataset. Our method is object agnostic, i.e., it presupposes the location of the object but not its identity. Starting from less than a hundred images of a whole object, we can generate several thousands images simulating visually diverse cuts of different coarseness. We use our synthetic data to train a model based on UNet and test it on real images showing coarsely/finely cut objects.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Artificial Intelligence in Games
MethodsFocus
