Towards Learning to Perceive and Reason About Liquids

TL;DR

This paper explores deep learning methods for liquid perception and reasoning in robotics, demonstrating that LSTM networks outperform other models in detecting and tracking liquids across multiple frames.

Contribution

It introduces the application of fully-convolutional neural networks, including LSTM, for liquid perception, a novel approach in robotics research.

Findings

LSTM networks outperform other models in liquid detection and tracking.

Aggregating data over multiple frames improves detection accuracy.

LSTM-based models show potential for robotic liquid handling.

Abstract

Recent advances in AI and robotics have claimed many incredible results with deep learning, yet no work to date has applied deep learning to the problem of liquid perception and reasoning. In this paper, we apply fully-convolutional deep neural networks to the tasks of detecting and tracking liquids. We evaluate three models: a single-frame network, multi-frame network, and a LSTM recurrent network. Our results show that the best liquid detection results are achieved when aggregating data over multiple frames and that the LSTM network outperforms the other two in both tasks. This suggests that LSTM-based neural networks have the potential to be a key component for enabling robots to handle liquids using robust, closed-loop controllers.