Extracting Insights from Large-Scale Telematics Data for ITS Applications: Lessons and Recommendations

TL;DR

This paper develops a data processing pipeline and visualization tools for large-scale telematics data, providing insights into trip behavior and roadway speeds to aid transportation planning and vehicle manufacturers.

Contribution

It introduces a scalable data analysis framework, an open data repository, and visualization tools for telematics data, addressing standardization and processing challenges.

Findings

Analyzed 1.4 billion miles of telematics data.

Created an open repository of trip and roadway summaries.

Identified key challenges and provided recommendations for data processing.

Abstract

Over 90% of new vehicles in the United States now collect and transmit telematics data. Similar trends are seen in other developed countries. Transportation planners have previously utilized telematics data in various forms, but its current scale offers significant new opportunities in traffic measurement, classification, planning, and control. Despite these opportunities, the enormous volume of data and lack of standardization across manufacturers necessitates a clearer understanding of the data and improved data processing methods for extracting actionable insights. This paper takes a step towards addressing these needs through four primary objectives. First, a data processing pipeline was built to efficiently analyze 1.4 billion miles (120 million trips) of telematics data collected in Virginia between August 2021 and August 2022. Second, an open data repository of trip and roadway segment level summaries was created. Third, interactive visualization tools were designed to extract insights from these data about trip-taking behavior and the speed profiles of roadways. Finally, major challenges that were faced during processing this data are summarized and recommendations to overcome them are provided. This work will help manufacturers collecting the data and transportation professionals using the data to develop a better understanding of the possibilities and major pitfalls to avoid.