Joint Prediction of Audio Event and Annoyance Rating in an Urban Soundscape by Hierarchical Graph Representation Learning

TL;DR

This paper introduces a hierarchical graph learning method that jointly predicts audio events and human annoyance ratings in urban soundscapes, linking objective sound data with subjective perception to improve environmental understanding.

Contribution

It presents a novel hierarchical graph representation learning approach that connects audio event features with annoyance ratings, capturing multi-grain semantic relations for better prediction.

Findings

Effective integration of audio events and annoyance ratings.

Improved prediction accuracy for soundscape perception.

Successful modeling of multi-grain semantic relations.

Abstract

Sound events in daily life carry rich information about the objective world. The composition of these sounds affects the mood of people in a soundscape. Most previous approaches only focus on classifying and detecting audio events and scenes, but may ignore their perceptual quality that may impact humans' listening mood for the environment, e.g. annoyance. To this end, this paper proposes a novel hierarchical graph representation learning (HGRL) approach which links objective audio events (AE) with subjective annoyance ratings (AR) of the soundscape perceived by humans. The hierarchical graph consists of fine-grained event (fAE) embeddings with single-class event semantics, coarse-grained event (cAE) embeddings with multi-class event semantics, and AR embeddings. Experiments show the proposed HGRL successfully integrates AE with AR for AEC and ARP tasks, while coordinating the relations between cAE and fAE and further aligning the two different grains of AE information with the AR.