Data streaming platform for crowd-sourced vehicle dataset generation

TL;DR

This paper presents an edge-cloud data platform for vehicle data collection and processing, addressing key challenges like privacy and interoperability, and evaluates its performance with different data types and connectivity technologies.

Contribution

It introduces a novel edge-cloud platform for heterogeneous vehicle data, analyzing performance limits and connectivity impacts to optimize data processing in automotive IoT.

Findings

Latency drops to 33ms with 5G connectivity using pipelining.

Performance varies with data type and connectivity technology.

Guidelines provided for processing assets to prevent bottlenecks.

Abstract

Vehicles are sophisticated machines equipped with sensors that provide real-time data for onboard driving assistance systems. Due to the wide variety of traffic, road, and weather conditions, continuous system enhancements are essential. Connectivity allows vehicles to transmit previously unknown data, expanding datasets and accelerating the development of new data models. This enables faster identification and integration of novel data, improving system reliability and reducing time to market. Data Spaces aim to create a data-driven, interconnected, and innovative data economy, where edge and cloud infrastructures support a virtualised IoT platform that connects data sources and development servers. This paper proposes an edge-cloud data platform to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy. The paper also evaluates the data platform's performance limits for text, image, and video data workloads, examines the impact of connectivity technologies, and assesses latencies. The results show that latencies drop to 33ms with 5G connectivity when pipelining data to consuming applications hosted at the edge, compared to around 77ms when crossing both edge and cloud processing infrastructures. The results offer guidance on the necessary processing assets to avoid bottlenecks in car data platforms.