Sage Deer: A Super-Aligned Driving Generalist Is Your Copilot

TL;DR

Sage Deer is a versatile, personalized driving agent that adapts to user preferences, understands multi-modal inputs, and enhances safety and comfort in intelligent driving cockpits.

Contribution

It introduces a super-aligned, generalist driving agent capable of multi-modal reasoning and personalized responses, supported by a large-scale benchmark dataset.

Findings

Achieves personalized reactions based on user preferences

Understands multi-view and multi-mode inputs for decision-making

Demonstrates high accuracy on a new large-scale benchmark

Abstract

The intelligent driving cockpit, an important part of intelligent driving, needs to match different users' comfort, interaction, and safety needs. This paper aims to build a Super-Aligned and GEneralist DRiving agent, SAGE DeeR. Sage Deer achieves three highlights: (1) Super alignment: It achieves different reactions according to different people's preferences and biases. (2) Generalist: It can understand the multi-view and multi-mode inputs to reason the user's physiological indicators, facial emotions, hand movements, body movements, driving scenarios, and behavioral decisions. (3) Self-Eliciting: It can elicit implicit thought chains in the language space to further increase generalist and super-aligned abilities. Besides, we collected multiple data sets and built a large-scale benchmark. This benchmark measures the deer's perceptual decision-making ability and the super alignment's accuracy.