Extraction of Discrete Spectra Modes from Video Data Using a Deep Convolutional Koopman Network

TL;DR

This paper introduces a deep convolutional Koopman network that automatically extracts discrete spectral modes from video data, enabling analysis of complex dynamical systems through observable variables.

Contribution

It presents a novel deep convolutional Koopman network capable of identifying system modes directly from video data, expanding applicability to observable dynamical systems.

Findings

Successfully identifies underlying modes in video-observed systems

Robustly handles large embedding spaces

Uses a simple masking procedure for modal disaggregation

Abstract

Recent deep learning extensions in Koopman theory have enabled compact, interpretable representations of nonlinear dynamical systems which are amenable to linear analysis. Deep Koopman networks attempt to learn the Koopman eigenfunctions which capture the coordinate transformation to globally linearize system dynamics. These eigenfunctions can be linked to underlying system modes which govern the dynamical behavior of the system. While many related techniques have demonstrated their efficacy on canonical systems and their associated state variables, in this work the system dynamics are observed optically (i.e. in video format). We demonstrate the ability of a deep convolutional Koopman network (CKN) in automatically identifying independent modes for dynamical systems with discrete spectra. Practically, this affords flexibility in system data collection as the data are easily obtainable observable variables. The learned models are able to successfully and robustly identify the underlying modes governing the system, even with a redundantly large embedding space. Modal disaggregation is encouraged using a simple masking procedure. All of the systems analyzed in this work use an identical network architecture.