Lang2Lift: A Language-Guided Autonomous Forklift System for Outdoor Industrial Pallet Handling

TL;DR

Lang2Lift is an innovative autonomous forklift system that uses natural language instructions and perception modules to identify, locate, and pick up pallets in complex outdoor environments, enhancing flexibility and practicality.

Contribution

The paper introduces a novel language-guided perception and control system for outdoor forklifts, enabling flexible pallet handling in unstructured, real-world scenarios.

Findings

Successful deployment on outdoor forklift platform

Effective language-grounded perception for pallet identification

Accurate pose estimation for manipulation tasks

Abstract

Automating pallet handling in outdoor logistics and construction environments remains challenging due to unstructured scenes, variable pallet configurations, and changing environmental conditions. In this paper, we present Lang2Lift, an end-to-end language-guided autonomous forklift system designed to support practical pallet pick-up operations in real-world outdoor settings. The system enables operators to specify target pallets using natural language instructions, allowing flexible selection among multiple pallets with different loads and spatial arrangements. Lang2Lift integrates foundation-model-based perception modules with motion planning and control in a closed-loop autonomy pipeline. Language-grounded visual perception is used to identify and segment target pallets, followed by 6D pose estimation and geometric refinement to generate manipulation-feasible insertion poses. The resulting pose estimates are directly coupled with the forklift planning and control modules to execute fully autonomous pallet pick-up maneuvers. We deploy and evaluate the proposed system on the ADAPT autonomous outdoor forklift platform across diverse real-world scenarios, including cluttered scenes, variable lighting, and different payload configurations. Tolerance-based pose evaluation further indicates accuracy sufficient for successful fork insertion. Timing and failure analyses highlight key deployment trade-offs and practical limitations, providing insights into integrating language-guided perception within industrial automation systems. Video demonstrations are available at https://eric-nguyen1402.github.io/lang2lift.github.io/