Can we still use PEAQ? A Performance Analysis of the ITU Standard for the Objective Assessment of Perceived Audio Quality

TL;DR

This paper evaluates the current performance of the PEAQ method for audio quality assessment, identifies its limitations with modern codecs, and suggests improvements based on experimental comparisons with newer objective measures.

Contribution

The study provides a comprehensive analysis of PEAQ's performance with modern audio signals and proposes updates to enhance its accuracy and reliability.

Findings

PEAQ's disturbance loudness model performs comparably to state-of-the-art measures.

Updating the mapping of Model Output Values improves performance.

PEAQ shows limitations with newer coding technologies like bandwidth extension.

Abstract

The Perceptual Evaluation of Audio Quality (PEAQ) method as described in the International Telecommunication Union (ITU) recommendation ITU-R BS.1387 has been widely used for computationally estimating the quality of perceptually coded audio signals without the need for extensive subjective listening tests. However, many reports have highlighted clear limitations of the scheme after the end of its standardization, particularly involving signals coded with newer technologies such as bandwidth extension or parametric multi-channel coding. Until now, no other method for measuring the quality of both speech and audio signals has been standardized by the ITU. Therefore, a further investigation of the causes for these limitations would be beneficial to a possible update of said scheme. Our experimental results indicate that the performance of PEAQ's model of disturbance loudness is still as good as (and sometimes superior to) other state-of-the-art objective measures, albeit with varying performance depending on the type of degraded signal content (i.e. speech or music). This finding evidences the need for an improved cognitive model. In addition, results indicate that an updated mapping of Model Output Values (MOVs) to PEAQ's Distortion Index (DI) based on newer training data can greatly improve performance. Finally, some suggestions for the improvement of PEAQ are provided based on the reported results and comparison to other systems.