How Should We Evaluate Synthesized Environmental Sounds

TL;DR

This paper explores how to evaluate synthesized environmental sounds, proposing a subjective evaluation method and demonstrating that subjective and objective assessments often yield different results.

Contribution

It introduces a novel subjective evaluation methodology for environmental sound synthesis and compares it with conventional methods, highlighting the importance of both evaluation types.

Findings

Subjective evaluations often differ from objective evaluations.

Both evaluation types are necessary for comprehensive assessment.

Proposed methodology effectively captures perceptual quality.

Abstract

Although several methods of environmental sound synthesis have been proposed, there has been no discussion on how synthesized environmental sounds should be evaluated. Only either subjective or objective evaluations have been conducted in conventional evaluations, and it is not clear what type of evaluation should be carried out. In this paper, we investigate how to evaluate synthesized environmental sounds. We also propose a subjective evaluation methodology to evaluate whether the synthesized sound appropriately represents the information input to the environmental sound synthesis system. In our experiments, we compare the proposed and conventional evaluation methods and show that the results of subjective evaluations tended to differ from those of objective evaluations. From these results, we conclude that it is necessary to conduct not only objective evaluation but also subjective evaluation.