roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphs

TL;DR

roadscene2vec is an open-source toolkit that facilitates the extraction, embedding, visualization, and analysis of road scene-graphs, advancing research in traffic safety and autonomous driving applications.

Contribution

It introduces a comprehensive, configurable tool for generating and analyzing road scene-graphs, supporting various models and applications like risk assessment and collision prediction.

Findings

Effective scene-graph extraction from videos and simulations.

Demonstrated utility in risk assessment and collision prediction.

Provides tools for transfer learning and explainability analysis.

Abstract

Recently, road scene-graph representations used in conjunction with graph learning techniques have been shown to outperform state-of-the-art deep learning techniques in tasks including action classification, risk assessment, and collision prediction. To enable the exploration of applications of road scene-graph representations, we introduce roadscene2vec: an open-source tool for extracting and embedding road scene-graphs. The goal of roadscene2vec is to enable research into the applications and capabilities of road scene-graphs by providing tools for generating scene-graphs, graph learning models to generate spatio-temporal scene-graph embeddings, and tools for visualizing and analyzing scene-graph-based methodologies. The capabilities of roadscene2vec include (i) customized scene-graph generation from either video clips or data from the CARLA simulator, (ii) multiple configurable spatio-temporal graph embedding models and baseline CNN-based models, (iii) built-in functionality for using graph and sequence embeddings for risk assessment and collision prediction applications, (iv) tools for evaluating transfer learning, and (v) utilities for visualizing scene-graphs and analyzing the explainability of graph learning models. We demonstrate the utility of roadscene2vec for these use cases with experimental results and qualitative evaluations for both graph learning models and CNN-based models. roadscene2vec is available at https://github.com/AICPS/roadscene2vec.