Improving Real-time Score Following in Opera by Combining Music with Lyrics Tracking
Charles Brazier, Gerhard Widmer

TL;DR
This paper introduces a novel opera tracking pipeline combining music and lyrics trackers to improve accuracy and robustness in real-time score following, addressing challenges posed by the genre's acoustic complexity.
Contribution
The paper presents a new combined tracking approach that integrates music and lyrics tracking to enhance real-time opera score following performance.
Findings
Improved accuracy in opera tracking demonstrated on Don Giovanni.
Enhanced robustness of tracking in complex acoustic environments.
Effective correction of music tracker errors using lyrics information.
Abstract
Fully automatic opera tracking is challenging because of the acoustic complexity of the genre, combining musical and linguistic information (singing, speech) in complex ways. In this paper, we propose a new pipeline for complete opera tracking. The pipeline is based on two trackers. A music tracker that has proven to be effective at tracking orchestral parts, will lead the tracking process. In addition, a lyrics tracker, that has recently been shown to reliably track the lyrics of opera songs, will correct the music tracker when tracking parts that have a text dominance over the music. We will demonstrate the efficiency of this method on the opera Don Giovanni, showing that this technique helps improving accuracy and robustness of a complete opera tracker.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Music Technology and Sound Studies · Human Motion and Animation
