Pour me a drink: Robotic Precision Pouring Carbonated Beverages into Transparent Containers

TL;DR

This paper introduces an autonomous robotic system capable of precisely pouring various liquids, including carbonated beverages, into different containers using only RGB vision and integrating ChatGPT for user interaction.

Contribution

The work presents a zero-shot, vision-based pouring pipeline that handles diverse liquids and containers without additional training, enhanced by ChatGPT for user-friendly operation.

Findings

Successfully pours multiple beverage types into various containers

Achieves high accuracy using only RGB input and pre-trained segmentation models

Demonstrates seamless human-robot interaction via ChatGPT

Abstract

With the growing emphasis on the development and integration of service robots within household environments, we will need to endow robots with the ability to reliably pour a variety of liquids. However, liquid handling and pouring is a challenging task due to the complex dynamics and varying properties of different liquids, the exacting precision required to prevent spills and ensure accurate pouring, and the necessity for robots to adapt seamlessly to a multitude of containers in real-world scenarios. In response to these challenges, we propose a novel autonomous robotics pipeline that empowers robots to execute precision pouring tasks, encompassing both carbonated and non-carbonated liquids, as well as opaque and transparent liquids, into a variety of transparent containers. Our proposed approach maximizes the potential of RGB input alone, achieving zero-shot capability by harnessing existing pre-trained vision segmentation models. This eliminates the need for additional data collection, manual image annotations, or extensive training. Furthermore, our work integrates ChatGPT, facilitating seamless interaction between individuals without prior expertise in robotics and our pouring pipeline, this integration enables users to effortlessly request and execute pouring actions. Our experiments demonstrate the pipeline's capability to successfully pour a diverse range of carbonated and non-carbonated beverages into containers of varying sizes, relying solely on visual input.