FruitsMusic: A Real-World Corpus of Japanese Idol-Group Songs

TL;DR

FruitsMusic is a detailed, annotated corpus of Japanese idol-group songs designed to improve singer diarization and music information retrieval techniques in real-world scenarios.

Contribution

This paper introduces FruitsMusic, a novel, richly annotated corpus of Japanese idol-group songs, and evaluates current diarization methods on this challenging real-world dataset.

Findings

Current methods show limited accuracy on FruitsMusic data.

Human performance exceeds automated diarization results.

FruitsMusic enables development of advanced singer recognition techniques.

Abstract

This study presents FruitsMusic, a metadata corpus of Japanese idol-group songs in the real world, precisely annotated with who sings what and when. Japanese idol-group songs, vital to Japanese pop culture, feature a unique vocal arrangement style, where songs are divided into several segments, and a specific individual or multiple singers are assigned to each segment. To enhance singer diarization methods for recognizing such structures, we constructed FruitsMusic as a resource using 40 music videos of Japanese idol groups from YouTube. The corpus includes detailed annotations, covering songs across various genres, division and assignment styles, and groups ranging from 4 to 9 members. FruitsMusic also facilitates the development of various music information retrieval techniques, such as lyrics transcription and singer identification, benefiting not only Japanese idol-group songs but also a wide range of songs featuring single or multiple singers from various cultures. This paper offers a comprehensive overview of FruitsMusic, including its creation methodology and unique characteristics compared to conversational speech. Additionally, this paper evaluates the efficacy of current methods for singer embedding extraction and diarization in challenging real-world conditions using FruitsMusic. Furthermore, this paper examines potential improvements in automatic diarization performance through evaluating human performance.