World-Model-Based Control for Industrial box-packing of Multiple Objects using NewtonianVAE

TL;DR

This paper introduces a vision-based control model using NewtonianVAE for industrial box-packing, enabling robots to accurately place multiple objects sequentially with high success rates, adaptable to new products without retraining.

Contribution

The study presents a novel in-hand-view-sensitive NewtonianVAE model that handles sequential object placement tasks with a single training, outperforming existing methods in industrial packing.

Findings

Achieved 100% success rate in real-world box-packing tasks.

Model trained on one object type generalizes to sequential tasks.

Outperforms state-of-the-art approaches in accuracy and adaptability.

Abstract

The process of industrial box-packing, which involves the accurate placement of multiple objects, requires high-accuracy positioning and sequential actions. When a robot is tasked with placing an object at a specific location with high accuracy, it is important not only to have information about the location of the object to be placed, but also the posture of the object grasped by the robotic hand. Often, industrial box-packing requires the sequential placement of identically shaped objects into a single box. The robot's action should be determined by the same learned model. In factories, new kinds of products often appear and there is a need for a model that can easily adapt to them. Therefore, it should be easy to collect data to train the model. In this study, we designed a robotic system to automate real-world industrial tasks, employing a vision-based learning control model. We propose in-hand-view-sensitive Newtonian variational autoencoder (ihVS-NVAE), which employs an RGB camera to obtain in-hand postures of objects. We demonstrate that our model, trained for a single object-placement task, can handle sequential tasks without additional training. To evaluate efficacy of the proposed model, we employed a real robot to perform sequential industrial box-packing of multiple objects. Results showed that the proposed model achieved a 100% success rate in industrial box-packing tasks, thereby outperforming the state-of-the-art and conventional approaches, underscoring its superior effectiveness and potential in industrial tasks.