Standardized Evaluation of Fetal Phonocardiography Processing Methods

TL;DR

This paper evaluates various fetal phonocardiography processing methods using a standardized benchmarking platform, revealing no single best method and emphasizing the need for further standardization in evaluation practices.

Contribution

It introduces a standardized benchmarking approach for fetal heart sound detection methods, enabling fair comparison and highlighting the variability in method performance.

Findings

No method outperforms others in all tests

Simpler methods can perform comparably to complex ones

Best models achieved high accuracy in heart sound detection and reasonable error in heart rate estimation

Abstract

Motivation. Phonocardiography can give access to the fetal heart rate as well as direct heart sound data, and is entirely passive, using no radiation of any kind. Approach. We discuss the currently available methods for fetal heart sound detection and heart rate estimation and compare them using a common benchmarking platform and a pre-selected testing dataset. Compared to previous reviews, we evaluated the discussed methods in a standardized manner for a fair comparison. Our tests included tolerance-based detection accuracy, error rates for label insertions, deletions, and substitutions, and statistical measures for heart rate mean square error. Results. Based on our results, there is no definite best method that can achieve the highest scores in all of the tests, and simpler methods could perform comparably to more complex ones. The best model for first heart sound detection achieved 97.6% F1-score, 97.4% positive predictive value, and 12.2+-8.0 ms mean absolute error. In terms of second heart sound detection the best model had 91.4% F1-score, 91.3% positive predictive value, and 17.3+-12.2 ms mean absolute error. For fetal heart rate a 0.644 mean square error was achieved by the best method. Significance. Our main conclusion is that further standardization is required in fetal heart rate and heart sound detection method evaluation. The tests and algorithm implementations are openly available at: https://github.com/mulkr/standard-fpcg-evaluation.