Language-Based Swarm Perception: Decentralized Person Re-Identification via Natural Language Descriptions

TL;DR

This paper presents a decentralized person re-identification method for robot swarms that uses natural language descriptions instead of visual embeddings, enabling interpretable and collaborative identification.

Contribution

The paper introduces a novel language-based perception approach for swarm robots, replacing traditional visual embeddings with human-readable descriptions for decentralized person re-identification.

Findings

Competitive identity consistency performance

Enhanced interpretability of swarm perception

Potential for natural-language querying and transparency

Abstract

We introduce a method for decentralized person re-identification in robot swarms that leverages natural language as the primary representational modality. Unlike traditional approaches that rely on opaque visual embeddings -- high-dimensional feature vectors extracted from images -- the proposed method uses human-readable language to represent observations. Each robot locally detects and describes individuals using a vision-language model (VLM), producing textual descriptions of appearance instead of feature vectors. These descriptions are compared and clustered across the swarm without centralized coordination, allowing robots to collaboratively group observations of the same individual. Each cluster is distilled into a representative description by a language model, providing an interpretable, concise summary of the swarm's collective perception. This approach enables natural-language querying, enhances transparency, and supports explainable swarm behavior. Preliminary experiments demonstrate competitive performance in identity consistency and interpretability compared to embedding-based methods, despite current limitations in text similarity and computational load. Ongoing work explores refined similarity metrics, semantic navigation, and the extension of language-based perception to environmental elements. This work prioritizes decentralized perception and communication, while active navigation remains an open direction for future study.