Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving

TL;DR

LeapAD introduces a dual-process, human-inspired approach to autonomous driving that combines analytical reasoning with heuristic processing, enabling continuous learning and improved adaptability in complex environments with less labeled data.

Contribution

The paper presents LeapAD, a novel dual-process framework that emulates human cognition for autonomous driving, integrating continuous learning, reflection, and transfer of knowledge between analytical and heuristic modules.

Findings

LeapAD outperforms camera-only methods in CARLA tests.

Requires 1-2 orders of magnitude less labeled data.

Heuristic process inherits knowledge from GPT-4 powered analytic process.

Abstract

Autonomous driving has advanced significantly due to sensors, machine learning, and artificial intelligence improvements. However, prevailing methods struggle with intricate scenarios and causal relationships, hindering adaptability and interpretability in varied environments. To address the above problems, we introduce LeapAD, a novel paradigm for autonomous driving inspired by the human cognitive process. Specifically, LeapAD emulates human attention by selecting critical objects relevant to driving decisions, simplifying environmental interpretation, and mitigating decision-making complexities. Additionally, LeapAD incorporates an innovative dual-process decision-making module, which consists of an Analytic Process (System-II) for thorough analysis and reasoning, along with a Heuristic Process (System-I) for swift and empirical processing. The Analytic Process leverages its logical reasoning to accumulate linguistic driving experience, which is then transferred to the Heuristic Process by supervised fine-tuning. Through reflection mechanisms and a growing memory bank, LeapAD continuously improves itself from past mistakes in a closed-loop environment. Closed-loop testing in CARLA shows that LeapAD outperforms all methods relying solely on camera input, requiring 1-2 orders of magnitude less labeled data. Experiments also demonstrate that as the memory bank expands, the Heuristic Process with only 1.8B parameters can inherit the knowledge from a GPT-4 powered Analytic Process and achieve continuous performance improvement. Project page: https://pjlab-adg.github.io/LeapAD.