Music Proofreading with RefinPaint: Where and How to Modify Compositions given Context

TL;DR

RefinPaint is an iterative method that enhances music generation and proofreading by identifying weak elements and guiding resampling, improving composition quality for both AI and human creators.

Contribution

The paper introduces RefinPaint, a novel iterative inpainting and proofreading technique that leverages feedback to refine music compositions more effectively.

Findings

Effective inpainting and proofreading demonstrated

Improves automatic music generation quality

Assists amateur composers in refining their work

Abstract

Autoregressive generative transformers are key in music generation, producing coherent compositions but facing challenges in human-machine collaboration. We propose RefinPaint, an iterative technique that improves the sampling process. It does this by identifying the weaker music elements using a feedback model, which then informs the choices for resampling by an inpainting model. This dual-focus methodology not only facilitates the machine's ability to improve its automatic inpainting generation through repeated cycles but also offers a valuable tool for humans seeking to refine their compositions with automatic proofreading. Experimental results suggest RefinPaint's effectiveness in inpainting and proofreading tasks, demonstrating its value for refining music created by both machines and humans. This approach not only facilitates creativity but also aids amateur composers in improving their work.