Virtual Temporal Samples for Recurrent Neural Networks: applied to semantic segmentation in agriculture

TL;DR

This paper introduces a method to generate virtual temporal samples from still images to train recurrent neural networks for agricultural semantic segmentation, reducing the need for laborious video annotation.

Contribution

The authors propose a novel virtual data augmentation technique that enables training RNNs for temporal segmentation without labeled video sequences, specifically applied to agriculture.

Findings

Improved segmentation accuracy by 4.6% on sweet pepper dataset.

Enhanced performance by 4.9% on sugar beet dataset.

Effective temporal classification without synthetic data or extensive labeling.

Abstract

This paper explores the potential for performing temporal semantic segmentation in the context of agricultural robotics without temporally labelled data. We achieve this by proposing to generate virtual temporal samples from labelled still images. By exploiting the relatively static scene and assuming that the robot (camera) moves we are able to generate virtually labelled temporal sequences with no extra annotation effort. Normally, to train a recurrent neural network (RNN), labelled samples from a video (temporal) sequence are required which is laborious and has stymied work in this direction. By generating virtual temporal samples, we demonstrate that it is possible to train a lightweight RNN to perform semantic segmentation on two challenging agricultural datasets. Our results show that by training a temporal semantic segmenter using virtual samples we can increase the performance by an absolute amount of $4.6$ and $4.9$ on sweet pepper and sugar beet datasets, respectively. This indicates that our virtual data augmentation technique is able to accurately classify agricultural images temporally without the use of complicated synthetic data generation techniques nor with the overhead of labelling large amounts of temporal sequences.