miditok: A Python package for MIDI file tokenization

TL;DR

Miditok is an open-source Python library that provides flexible and extended tokenization of symbolic music, facilitating the use of language models like Transformers for music generation, modeling, and transcription.

Contribution

It introduces a unified API for various popular music tokenizations, making music encoding and decoding more accessible and extensible for researchers and developers.

Findings

Supports multiple tokenization schemes with a unified API

Enhances flexibility and extensibility for symbolic music processing

Facilitates integration of language models in music applications

Abstract

Recent progress in natural language processing has been adapted to the symbolic music modality. Language models, such as Transformers, have been used with symbolic music for a variety of tasks among which music generation, modeling or transcription, with state-of-the-art performances. These models are beginning to be used in production products. To encode and decode music for the backbone model, they need to rely on tokenizers, whose role is to serialize music into sequences of distinct elements called tokens. MidiTok is an open-source library allowing to tokenize symbolic music with great flexibility and extended features. It features the most popular music tokenizations, under a unified API. It is made to be easily used and extensible for everyone.