A Proposal for Foley Sound Synthesis Challenge

TL;DR

This paper proposes a challenge to advance automatic foley sound synthesis by establishing standardized evaluation methods, aiming to stimulate research and development in machine-assisted sound effect generation.

Contribution

It introduces a structured challenge framework for foley sound synthesis, including task definition, dataset, and evaluation criteria, to promote community engagement and progress.

Findings

Designed a unified evaluation protocol

Outlined dataset and task specifications

Encouraged community participation in sound synthesis research

Abstract

"Foley" refers to sound effects that are added to multimedia during post-production to enhance its perceived acoustic properties, e.g., by simulating the sounds of footsteps, ambient environmental sounds, or visible objects on the screen. While foley is traditionally produced by foley artists, there is increasing interest in automatic or machine-assisted techniques building upon recent advances in sound synthesis and generative models. To foster more participation in this growing research area, we propose a challenge for automatic foley synthesis. Through case studies on successful previous challenges in audio and machine learning, we set the goals of the proposed challenge: rigorous, unified, and efficient evaluation of different foley synthesis systems, with an overarching goal of drawing active participation from the research community. We outline the details and design considerations of a foley sound synthesis challenge, including task definition, dataset requirements, and evaluation criteria.