Iola Walker: A Mobile Footfall Detection System for Music Composition

TL;DR

This paper presents Iola Walker, an Android-based footfall detection system using LSTM neural networks trained on IMU data to enhance music composition through wearable sensor input.

Contribution

It introduces a novel mobile footfall detection system that integrates hardware and software for real-time music-related applications, with open-source code for further development.

Findings

Successful live inference of footfalls on Android devices

Effective training of LSTM on IMU data for footfall detection

Open-source implementation for community use and improvement

Abstract

This outing is part of a larger music technology research project. The objective is to find a way to enhance music using hardware and software. This is the documentation for the Whimsical first part of the research project: it's an android app that detects a wearer's footfalls by running live inference on an LSTM. The system works by getting data from an Mbient Labs IMU to a mobile app over bluetooth. After you move the .csv file to a computer with a GPU, you can use the python code to train an LSTM on that data. You then export the LSTM to the android app and can begin detecting footfalls. Feel free to download and experiment with the code. It's meant to be read and improved upon by you and your LLM codewriter of choice! https://github.com/willbjames/iolawalker