Classification and regression of trajectories rendered as images via 2D Convolutional Neural Networks

TL;DR

This paper explores how 2D CNNs can classify and regress trajectories rendered as images, analyzing how rendering parameters affect performance and addressing artifacts like information loss and spectral changes.

Contribution

It investigates the effectiveness of CNNs on synthetic trajectory images with various rendering modalities, highlighting the impact of image resolution and motion history.

Findings

Optimal image resolution depends on model depth and motion history.

Motion history encoding improves movement direction recognition.

Rendering artifacts can significantly affect CNN performance.

Abstract

Trajectories can be regarded as time-series of coordinates, typically arising from motile objects. Methods for trajectory classification are particularly important to detect different movement patterns, while methods for regression to compute motility metrics and forecasting. Recent advances in computer vision have facilitated the processing of trajectories rendered as images via artificial neural networks with 2d convolutional layers (CNNs). This approach leverages the capability of CNNs to learn spatial hierarchies of features from images, necessary to recognize complex shapes. Moreover, it overcomes the limitation of other machine learning methods that require input trajectories with a fixed number of points. However, rendering trajectories as images can introduce poorly investigated artifacts such as information loss due to the plotting of coordinates on a discrete grid, and spectral changes due to line thickness and aliasing. In this study, we investigate the effectiveness of CNNs for solving classification and regression problems from synthetic trajectories that have been rendered as images using different modalities. The parameters considered in this study include line thickness, image resolution, usage of motion history (color-coding of the temporal component) and anti-aliasing. Results highlight the importance of choosing an appropriate image resolution according to model depth and motion history in applications where movement direction is critical.