Unified Acoustic Representations for Screening Neurological and Respiratory Pathologies from Voice

TL;DR

This paper introduces MARVEL, a multitask learning framework that uses acoustic features to simultaneously detect nine neurological and respiratory disorders from voice, achieving high accuracy and outperforming existing models.

Contribution

The paper presents a novel unified multitask acoustic representation model for multi-condition voice-based health screening, leveraging cross-condition knowledge transfer and outperforming state-of-the-art methods.

Findings

Achieves an overall AUROC of 0.78 across nine disorders.

Performs exceptionally well on neurological disorders with AUROC up to 0.97.

Outperforms single-modal baselines and state-of-the-art models on most tasks.

Abstract

Voice-based health assessment offers unprecedented opportunities for scalable, non-invasive disease screening, yet existing approaches typically focus on single conditions and fail to leverage the rich, multi-faceted information embedded in speech. We present MARVEL (Multi-task Acoustic Representations for Voice-based Health Analysis), a privacy-conscious multitask learning framework that simultaneously detects nine distinct neurological, respiratory, and voice disorders using only derived acoustic features, eliminating the need for raw audio transmission. Our dual-branch architecture employs specialized encoders with task-specific heads sharing a common acoustic backbone, enabling effective cross-condition knowledge transfer. Evaluated on the large-scale Bridge2AI-Voice v2.0 dataset, MARVEL achieves an overall AUROC of 0.78, with exceptional performance on neurological disorders (AUROC = 0.89), particularly for Alzheimer's disease/mild cognitive impairment (AUROC = 0.97). Our framework consistently outperforms single-modal baselines by 5-19% and surpasses state-of-the-art self-supervised models on 7 of 9 tasks, while correlation analysis reveals that the learned representations exhibit meaningful similarities with established acoustic features, indicating that the model's internal representations are consistent with clinically recognized acoustic patterns. By demonstrating that a single unified model can effectively screen for diverse conditions, this work establishes a foundation for deployable voice-based diagnostics in resource-constrained and remote healthcare settings.

Tables

Table 1. Table 1. Participant and sample statistics for each binary classification task. P = positive class, N = negative class.

Disorder	Subtype Diagnosis	Train Participants (#S)		Test Participants (#S)		Total
		P	N	P	N	P	N
Respiratory	Airway Stenosis	42 (1239)	42 (1527)	9 (284)	9 (331)	51	51
	COPD	7 (251)	7 (241)	4 (130)	4 (133)	11	11
	Asthma	27 (905)	27 (870)	6 (214)	6 (194)	33	33
	Vocal Fold Paralysis	25 (875)	25 (891)	5 (166)	5 (155)	30	30
Voice	Laryngeal Cancer	6 (203)	6 (210)	2 (70)	2 (66)	8	8
	Benign Lesions of the Vocal Cord	19 (679)	19 (663)	5 (174)	5 (221)	24	24
	Spasmodic Dysphonia/Laryngeal Tremor	31 (1091)	31 (1056)	7 (235)	7 (321)	38	38
Neurological	Parkinson’s	51 (2284)	51 (1779)	10 (874)	10 (575)	61	61
	Alzheimer’s Disease (AD) and MCI	34 (1513)	34 (1094)	6 (480)	6 (306)	40	40

Table 2. Table 2. AUROC (mean ± \pm std over 5 runs) for each disorder category. Baseline models use single-task learning with individual modalities (MFCC, spectrogram, or handcrafted features).

Disorder Type	MLP	${\text{EN}}_m$	${\text{EN}}_s$	${\text{RN}}_m$	${\text{RN}}_s$	MARVEL
Respiratory	0.71 ± 0.15	0.73 ± 0.15	0.70 ± 0.12	0.73 ± 0.14	0.72 ± 0.16	0.74 ± 0.18
Voice	0.67 ± 0.09	0.66 ± 0.11	0.71 ± 0.09	0.63 ± 0.12	0.67 ± 0.07	0.76 ± 0.08
Neurological	0.80 ± 0.09	0.76 ± 0.01	0.77 ± 0.08	0.74 ± 0.06	0.73 ± 0.04	0.89 ± 0.11
All	0.71 ± 0.10	0.70 ± 0.11	0.71 ± 0.09	0.67 ± 0.11	0.69 ± 0.09	0.78 ± 0.14

Table 3. Table 3. AUROC comparison between single-task self-supervised baselines and the proposed multi-task learning (MTL) model across subtype disorders (mean ± std over 5 runs).

Subtype Task	Whisper (radford2023robust)	Wav2vec-BERT (barrault2023seamless)	M-CTC-T (lugosch2022pseudo)	MARVEL
Airway Stenosis	0.84 ± 0.02	0.80 ± 0.03	0.78 ± 0.03	0.89 ± 0.01
Asthma	0.53 ± 0.03	0.56 ± 0.03	0.53 ± 0.03	0.54 ± 0.03
Benign Lesions of the Vocal Cord	0.51 ± 0.03	0.64 ± 0.03	0.60 ± 0.03	0.70 ± 0.01
COPD	0.67 ± 0.04	0.74 ± 0.03	0.75 ± 0.03	0.79 ± 0.03
Laryngeal Cancer / Pre-cancerous	0.70 ± 0.03	0.69 ± 0.03	0.67 ± 0.03	0.71 ± 0.04
Alzheimer’s Disease (AD) and MCI	0.91 ± 0.02	0.89 ± 0.01	0.88 ± 0.02	0.97 ± 0.01
Spasmodic Dysphonia / Laryngeal Tremor	0.72 ± 0.02	0.73 ± 0.03	0.72 ± 0.02	0.77 ± 0.02
Vocal Fold Paralysis	0.83 ± 0.02	0.78 ± 0.03	0.76 ± 0.03	0.87 ± 0.01
Parkinson’s	0.79 ± 0.04	0.78 ± 0.04	0.74 ± 0.05	0.81 ± 0.06

Table 4. Table 4. AUROC (mean ± std over 5 runs) on each subtype disorder classification task. Baselines use single-task learning while MARVEL employs multi-task learning across all model-input combinations.

Subtype Task	${\text{EN}}_m$	${\text{EN}}_s$	${\text{RN}}_m$	${\text{RN}}_s$	MLP	MARVEL
Airway Stenosis	0.80 ± 0.03	0.80 ± 0.04	0.83 ± 0.03	0.85 ± 0.04	0.84 ± 0.02	0.89 ± 0.01
Alzheimer’s Disease (AD) and MCI	0.77 ± 0.01	0.83 ± 0.01	0.78 ± 0.01	0.75 ± 0.02	0.86 ± 0.01	0.97 ± 0.01
Asthma	0.55 ± 0.03	0.57 ± 0.03	0.56 ± 0.04	0.54 ± 0.01	0.55 ± 0.03	0.54 ± 0.03
Benign Lesions of the Vocal Cord	0.65 ± 0.02	0.67 ± 0.01	0.61 ± 0.02	0.62 ± 0.03	0.59 ± 0.05	0.70 ± 0.01
COPD	0.83 ± 0.03	0.72 ± 0.08	0.79 ± 0.05	0.76 ± 0.04	0.74 ± 0.09	0.79 ± 0.03
Laryngeal Cancer / Pre-cancerous	0.52 ± 0.06	0.60 ± 0.11	0.46 ± 0.14	0.59 ± 0.08	0.58 ± 0.12	0.71 ± 0.04
Parkinson’s	0.75 ± 0.02	0.71 ± 0.03	0.69 ± 0.04	0.70 ± 0.03	0.73 ± 0.01	0.81 ± 0.06
Spasmodic Dysphonia / Laryngeal Tremor	0.77 ± 0.01	0.80 ± 0.02	0.75 ± 0.03	0.75 ± 0.04	0.77 ± 0.01	0.77 ± 0.02
Vocal Fold Paralysis	0.66 ± 0.09	0.77 ± 0.02	0.68 ± 0.04	0.70 ± 0.14	0.73 ± 0.01	0.87 ± 0.01

Table 5. Table 5. Cross-disease acoustic features that frequently appear in top-5 SHAP rankings. Avg SHAP denotes the mean absolute SHAP value of the feature, computed only from the diseases in which it was ranked among the top-5 most important features.

Feature	Avg SHAP	Associated Disorders
slopeUV0-500_sma3nz_amean	0.031	Alzheimerś Disease (AD) and MCI, Spasmodic Dysphonia Laryngeal Tremor, Parkinson’s
mfcc2V_sma3nz_amean	0.012	Airway Stenosis, Spasmodic Dysphonia Laryngeal Tremor, Vocal Fold Paralysis
spectral_gravity	0.005	Airway Stenosis, Asthma, COPD
loudness_sma3_percentile80.0	0.004	Airway Stenosis, Asthma, Benign Lesions of the Vocal Cord
cepstral_peak_prominence_mean	0.017	Vocal Fold Paralysis, Parkinson’s
mean_intensity_db	0.004	Laryngeal Cancer, Parkinson’s
mean_b1_loc	0.003	Asthma, Laryngeal Cancer

Equations

$$f:({\mathbf{x}}^{\text{mfcc}},{\mathbf{x}}^{\text{spec}},k)\mapsto {\hat{y}}_k$$

$${\mathbf{h}}_{\text{mfcc}}=f_{\text{mfcc}}({\mathbf{X}}^{\text{mfcc}})\in {\mathbb{R}}^{D_{\text{emb,mfcc}}},{\mathbf{h}}_{\text{spec}}=f_{\text{spec}}({\mathbf{X}}^{\text{spec}})\in {\mathbb{R}}^{D_{\text{emb,spec}}}$$

$${\mathbf{h}}_{\text{fused}}=\text{Concat}({\mathbf{h}}_{\text{mfcc}},{\mathbf{h}}_{\text{spec}})\in {\mathbb{R}}^{(D_{\text{emb,mfcc}}+D_{\text{emb,spec}})}$$

$$\mathbf{z}=g({\mathbf{h}}_{\text{fused}})\in {\mathbb{R}}^{512}$$

$${\hat{y}}^{(k)}=\sigma ({\text{MLP}}^{(k)}(\mathbf{z}))=\sigma ({\mathbf{W}}^{(k)}{\mathbf{z}}_{\text{head\_in}}^{(k)}+{\mathbf{b}}^{(k)})$$

$${\mathcal{L}}^{(k)}=-\left(w_1^{(k)}{y}^{(k)}\log {\hat{y}}^{(k)}+w_0^{(k)}(1-y^{(k)})\log (1-{\hat{y}}^{(k)})\right)$$

$${\mathcal{L}}_{\text{total}}=\sum _{k=1}^K{\mathcal{L}}^{(k)}$$

Full text

Voice-based diagnostics, Multitask learning, Acoustic feature representations, Neurological and respiratory disorder detection

††ccs: Computing methodologies Multitask learning††ccs: Computing methodologies Machine learning††ccs: Applied computing Health care information systems††ccs: Human-centered computing Accessibility technologies

1. Introduction

The human voice, intrinsically natural, non-invasive, and readily collectable, is increasingly recognized as a potent and low-cost digital biomarker for health (fagherazzi2021voice; naserclaes2021speech). Acoustic properties embedded within speech—such as pitch, prosody, speaking rate, and loudness—extend beyond conveying linguistic and emotional states, offering valuable indicators of an individual’s cognitive, neurological, respiratory, and vocal well-being (luz2021detecting). These insights underpin the potential of voice analysis as a scalable modality for the early screening and monitoring of diverse conditions, including Parkinson’s disease (PD), mild cognitive impairment (MCI), airway-related diseases, and various voice disorders (tsanas2011novel).

The clinical manifestations of these conditions often leave discernible imprints on vocal production. For instance, respiratory ailments like airway stenosis can lead to abnormal breath control and altered vocal tract coordination (gelbard2016disease), while voice disorders such as vocal fold paralysis or structural lesions typically result in unstable vocal quality (roy2007toward). Neurological or cognitive conditions like PD or MCI can impact prosody and fluency, often yielding slowed or monotonous speech patterns (skodda2011vocal; martinez2021ten). Traditional diagnostic workflows frequently necessitate in-person visits to healthcare facilities or the use of cumbersome physiological sensors. In contrast, speech-based analysis presents a significantly lower-burden, remote-friendly alternative, amenable to passive or active collection during everyday interactions, thereby easing patient engagement and facilitating longitudinal monitoring (robin2020evaluation).

The proliferation of mobile devices equipped with local processing capabilities has paved the way for deploying speech-based diagnostic systems in real-world, privacy-sensitive settings. Crucially, raw speech recordings, which may contain sensitive personal information, do not need to be transmitted. Instead, acoustic representations like Mel-frequency cepstral coefficients (MFCCs) or log-Mel spectrograms can be extracted locally on-device (backstrom2023privacy). This on-edge processing paradigm safeguards patient privacy, promotes ethical data handling, and enables the scalable deployment of voice-based health technologies across diverse healthcare ecosystems.

In this work, we introduce a multitask disorder classification framework, MARVEL  (Multi-task Acoustic Representations for Voice-based Health Analysis), that operates exclusively on such derived speech features.An overview of the our framework is shown in Figure 1. Our model employs a dual-branch architecture with a shared acoustic backbone and task-specific binary classification heads, facilitating task-conditioned inference across a spectrum of voice-related disorders. This multitask learning (MTL) structure promotes beneficial feature sharing across related disease tasks, potentially enhancing generalization, especially for patients presenting with co-existing conditions. We train and rigorously evaluate our framework on the Bridge2AI-Voice v2.0 dataset (bensoussan2025bridge2ai; goldberger2000physiobank), a large-scale, ethically sourced repository of speech samples linked to diagnostic labels for neurological, respiratory, and vocal disorders. Notably, this dataset provides pre-extracted acoustic features and de-identified metadata, aligning perfectly with our privacy-conscious modeling approach that obviates the need for raw audio access. In addition, voice-based analysis lends itself to a wide range of real-world clinical applications. It can be integrated into remote monitoring systems to track patients with chronic or progressive conditions such as heart failure or neurodegenerative diseases. In triage settings, brief voice recordings may assist in early screening and prioritization of at-risk individuals. Moreover, such non-invasive and low-cost tools are particularly valuable in low-resource environments, where access to specialized clinicians or diagnostic infrastructure is limited. Together, these use cases underscore the practical value of speech as a scalable modality for population-level health assessment.

To complement our MARVEL  framework, we further investigate the clinical relevance of established acoustic biomarkers by conducting an auxiliary analysis examining the correlation between the learned deep embeddings from MARVEL  and traditional handcrafted acoustic features. Previous studies show that handcrafted features alone can offer reasonable discrimination across certain disorder subtypes, motivating their use as a reference point for model interpretation (low2020automated; vasquez2019convolutional). This post hoc analysis reveals measurable similarities between the two, providing additional interpretability into what our model has implicitly learned and showing that it captures clinically relevant vocal patterns in a richer and more nuanced manner than can be described by predefined acoustic features. Moreover, the better performance of MARVEL  compared to models trained directly on handcrafted features suggests that our model leverages these acoustic cues while also learning more complex and task-specific representations.

We summarize our primary contributions as follows:

• •

Multi-task and Dual-branch Learning for Speech-Based Disease Classification: We propose a multitask, dual-branch deep learning framework MARVEL  that operates solely on derived speech representations (MFCCs and spectrograms). This framework supports task-conditioned inference across multiple neurological, respiratory, and vocal disorders, enhancing generalization while adhering to privacy-by-design principles.

• •

Robust Benchmarking on the Bridge2AI-Voice Dataset: We establish a strong and reproducible benchmark on the Bridge2AI-Voice v2.0 dataset using only speech-derived features, without accessing raw audio. Our approach demonstrates significant performance across key clinical tasks (e.g., achieving an F1-score of 0.85 and AUROC of 0.94 for mild cognitive impairment), offering a valuable baseline for future research in privacy-conscious voice-based health assessment.

• •

Interpretability through Acoustic Feature Correlation: Building on the observation that handcrafted acoustic features alone can provide reasonable discrimination for several disorder subtypes, we conduct a post hoc analysis examining the correlation between learned deep embeddings from MARVEL  and these clinically grounded descriptors. This analysis reveals task-specific similarities between our model’s internal representations and established acoustic patterns, enhancing interpretability while also demonstrating that MARVEL  captures richer and more nuanced information beyond what handcrafted features explicitly encode.

2. Related Work

2.1. Acoustic Approaches to Vocal Health Assessment

The utility of voice as a non-invasive modality for diagnosing a wide array of conditions, spanning neurological, respiratory, and voice-related disorders, has garnered significant research interest in recent years (kim2023non; gomez2013characterizing; tong2022respiratory). Voice-based biomarkers are particularly compelling due to their inherent accessibility, low cost, and sensitivity to subtle alterations in motor and respiratory function. For instance, Parkinson’s disease (PD) frequently manifests through characteristic changes such as monotone prosody and imprecise articulation(brin1992movement; canter1963speech). Similarly, chronic respiratory diseases like Chronic Obstructive Pulmonary Disease (COPD) are often associated with distinct alterations in respiratory acoustics, potentially indicating deficits in breath control and upper airway coordination (gross2009coordination). Given the substantial global prevalence of such conditions—COPD alone affects over 300 million people globally (vogelmeier2017global), and neurological disorders like PD impact a rapidly growing aging population (aarsland2021parkinson)—there is an urgent and unmet need for scalable, remote, and non-invasive screening and monitoring tools. Consequently, voice analysis presents a timely and clinically relevant avenue to support early detection and ongoing management across diverse disease domains.

Initial explorations in this field predominantly relied on handcrafted acoustic features—such as jitter, shimmer, harmonic-to-noise ratio (HNR), and formants—coupled with traditional machine learning classifiers like Support Vector Machines (SVMs) and Gaussian Mixture Models (GMMs) (anagnostopoulos2015features; verde2018voice). While these seminal studies established foundational links between vocal anomalies and clinical conditions, they often faced challenges in terms of robustness and generalizability across diverse datasets and recording conditions. The advent of deep learning has revolutionized the field, enabling automatic feature extraction directly from acoustic representations like MFCCs and spectrograms. Convolutional Neural Networks (CNNs) and recurrent architectures, such as Bidirectional Long Short-Term Memory networks (BiLSTMs), have proven effective in modeling the complex spatial and temporal patterns inherent in these representations (Peng2023; alam2025revamping). More recently, large-scale self-supervised learning (SSL) models, pre-trained on vast amounts of unlabeled audio data, such as wav2vec 2.0 (baevski2020wav2vec) and Whisper (radford2023robust), have demonstrated remarkable performance in various speech processing tasks, including applications in medical audio analysis (Cai2024; violeta2022investigating).

To leverage complementary information from different acoustic views or modalities, fusion-based architectures have also been explored. For example, multibranch CNN or Temporal Convolutional Network (TCN) models have been applied to tasks like pathological breath sound classification (zhao2022automatic; kim2024real). However, many prior works employing fusion tend to use homogeneous encoder structures (e.g., identical CNN backbones for different inputs), potentially limiting their capacity to leverage architecture-specific inductive biases suited for distinct feature types. Furthermore, a common limitation in speech-based disease detection is the assumption of single-label classification, which overlooks the clinical reality that patients often exhibit overlapping symptoms or co-morbidities across multiple systems (e.g., concurrent vocal and respiratory issues).

Our work differentiates itself by proposing a structurally heterogeneous dual-branch fusion model. This model processes MFCCs and log-Mel spectrograms through distinct, specialized encoders—ResNet and EfficientNet, respectively. This architectural choice is informed by recent findings demonstrating the unique strengths of these backbones for specific clinical audio tasks: Yu et al. (yin2024sarnet) highlighted ResNet’s efficacy in capturing temporal patterns for dysarthria severity modeling via its residual connections, while Kumar et al. (kumar2024depression) showcased EfficientNet’s advantages in spatial modeling of log-Mel spectrograms for depression detection through compound scaling. By integrating these complementary encoder types within a unified framework, we aim to capture a richer, more comprehensive acoustic representation. Crucially, our model operates exclusively on these derived acoustic features, ensuring privacy-conscious deployment without reliance on raw audio, a critical consideration for clinical applications.

2.2. Multitask Learning in Clinical AI and Voice Diagnostics

Multitask learning (MTL) has emerged as a powerful paradigm for simultaneously modeling multiple clinically relevant targets, particularly when these tasks are correlated and can benefit from shared underlying feature representations (caruana1997multitask). In the broader healthcare domain, MTL has been successfully applied to diverse problems, including prediction tasks using electronic health records (EHRs), patient outcome modeling, and multimodal clinical reasoning (niu2025medical; ding2019effectiveness). Such models typically employ shared encoder layers (backbone) followed by task-specific output heads, often leading to improvements in sample efficiency, model robustness, and generalization performance due to the regularizing effect of shared knowledge.

In the context of voice-based diagnostics, however, the majority of existing models remain confined to single-task setups (Orozco2016PD; govindu2023early), often assuming that each speech recording corresponds to a singular disease label. This simplification does not adequately reflect real-world clinical scenarios where patients frequently present with symptoms spanning multiple conditions, such as co-occurring vocal and respiratory disorders. Single-task models may struggle to disentangle these complex, overlapping patterns. MTL offers a natural and effective solution by explicitly modeling shared vocal biomarkers across different tasks while simultaneously learning to retain task-specific nuances necessary for accurate discrimination.

While recent studies have begun to explore the application of MTL to pathological speech analysis, for instance, in jointly assessing voice quality and speech intelligibility (joshy2023dysarthria), several gaps remain. Many of these approaches still rely on homogeneous input types (e.g., using only spectrograms or only MFCCs for all tasks) or restrict their modeling scope to a single family of disorders (e.g., only neurological conditions). Furthermore, the full potential of leveraging complementary information from diverse acoustic representations within an MTL framework has not been thoroughly explored.

Our research extends this line of inquiry in two significant directions. Firstly, we adopt a dual-branch fusion architecture where MFCCs and spectrograms are processed independently by specialized backbone encoders before fusion. This design allows the model to harness complementary temporal and spectral information more effectively than homogeneous approaches. Secondly, we broaden the diagnostic scope to concurrently address a diverse set of respiratory, vocal, and neurological disorders within a single, unified MTL framework. A critical distinction of our work is its exclusive reliance on speech-derived acoustic features. Unlike many clinical AI models that integrate multimodal inputs such as EHR data or medical imaging, our approach is tailored for scalable, privacy-conscious diagnostics using readily accessible voice data, making it particularly well-suited for remote and resource-constrained settings.

3. Methodology

3.1. Problem Definition

Clinical speech recordings encapsulate a wealth of acoustic information that can serve as indicators for a diverse range of health conditions, including those affecting respiratory, cognitive, and vocal fold function. These recordings are typically elicited via structured vocalization tasks, such as sustained phonation (e.g., holding an /a/ sound) or the reading of standardized passages (e.g., the “Rainbow Passag”), which are designed to accentuate disorder-specific speech patterns and vocal anomalies.

We frame the diagnostic challenge as a multi-task binary classification problem. For any given speech sample, represented by its derived acoustic features, and a specific diagnostic task (e.g., detecting Airway Stenosis), the objective of our model is to predict the presence or absence of the corresponding clinical condition. While each diagnostic task possesses its own binary label space ($y\in\{0,1\}$), all tasks share the same input modalities derived from the speech signal.

To effectively address the inherent heterogeneity of acoustic cues across different disorders, we formulate our approach as task-conditional dual-modal learning. Formally, for each input sample $i$, we have:

• •

$\mathbf{x}_{i}^{\text{mfcc}}\in\mathbb{R}^{D_{M}\times T_{i}}$: The Mel-frequency cepstral coefficient (MFCC) representation, where $D_{M}$ is the number of MFCC coefficients and $T_{i}$ is the number of time frames for sample $i$.

• •

$\mathbf{x}_{i}^{\text{spec}}\in\mathbb{R}^{D_{S}\times T_{i}}$: The log-Mel spectrogram representation, where $D_{S}$ is the number of Mel filter banks and $T_{i}$ is the number of time frames.

• •

$k\in\{1,\dots,K\}$: The task index, where $K$ is the total number of distinct diagnostic conditions being modeled.

• •

$y_{i,k}\in\{0,1\}$: The binary ground-truth label for sample $i$ corresponding to task $k$.

The goal is to learn a function $f$:

[TABLE]

where $\hat{y}_{k}$ is the predicted probability of the presence of condition $k$. This formulation facilitates parameter sharing across related tasks through a common backbone while maintaining task-specific output layers, thereby enabling nuanced, task-conditioned predictions.

3.2. Dataset and Input Representations

Our study utilizes a curated subset of the Bridge2AI-Voice v2.0 dataset (bensoussan2025bridge2ai; goldberger2000physiobank), a large-scale, ethically sourced voice health database. This dataset encompasses over 19,000 voice recordings from 442 unique participants, collected across five clinical institutions in North America. Each participant performed a standardized set of prompted speech tasks, and these recordings are linked with clinical diagnoses. Critically, to safeguard participant privacy, the publicly accessible version of the dataset exclusively contains derived acoustic representations—including log-Mel spectrograms, MFCCs, and a variety of statistical voice descriptors—with all raw audio data excluded.

For our purpose, we defined nine distinct binary classification tasks, categorized into three major disorder groups:

1. (1)

Respiratory disorders: Airway Stenosis, Asthma, Chronic Obstructive Pulmonary Disease (COPD). 2. (2)

Voice disorders: Laryngeal Cancer, Vocal Cord Lesions (benign), Spasmodic Dysphonia/Laryngeal Tremor, Vocal Fold Paralysis. 3. (3)

Neurological disorders: Parkinson’s Disease (PD), Alzheimer’s Disease/Mild Cognitive Impairment (AD/MCI).

These specific tasks were selected based on several criteria: sufficient data availability within the Bridge2AI-Voice dataset, strong clinical grounding in prior voice-based research, and the potential for impactful clinical application.

For each binary classification task, positive samples were derived from participants clinically diagnosed with the target condition. Negative samples were carefully selected from participants diagnosed with disorders distinct from the target condition, ensuring diagnostic independence between tasks and mitigating the risk of label leakage. Each diagnostic condition was thus modeled as an independent binary classification task, with its own definition of positive and negative samples. When patients presented with multiple comorbid conditions, each diagnosis was treated as a separate binary decision, and tasks were not assumed to be mutually exclusive. As a result, a single voice recording could contribute to multiple task-specific datasets—serving as a positive sample for one condition and a negative or positive sample for others—without introducing methodological conflict. This formulation reflected the clinical reality of overlapping diagnoses and supported the multitask architecture’s ability to learn both shared and condition-specific acoustic patterns.

Detailed statistics regarding the distribution of participants and samples across each diagnostic category for training and testing splits are presented in Table 1. In total, our analytical dataset comprised 11,667 positively labeled recordings originating from 296 unique patients, covering over 49 distinct speech tasks performed by these individuals.

Each voice recording is processed into two primary time-frequency representations, which serve as inputs to our model: Log-Mel Spectrograms ($\mathbf{x}^{\text{spec}}$): These capture the spectral energy distribution of the speech signal over time. The resulting power spectrum is then warped onto the Mel scale and log-transformed. For our model, these are shaped as $\mathbb{R}^{1\times T\times F}$, where $F$ is the number of Mel filters (e.g., 128) and $T$ is the number of time frames. Mel-Frequency Cepstral Coefficients (MFCCs, $\mathbf{x}^{\text{mfcc}}$): MFCCs provide a compact representation of the spectral envelope, effectively capturing smooth vocal tract dynamics. They are derived by applying a Discrete Cosine Transform (DCT) to the log-Mel spectrum. We use 60 MFCCs (including C0), shaped as $\mathbb{R}^{1\times T\times 60}$.

To offer a qualitative insight into the acoustic diversity across different conditions, Figure 2 visualizes log-Mel spectrograms for the nine targeted voice-related disorders. Each panel corresponds to a speech sample from a single participant performing a specific task, highlighting how disease-specific vocal characteristics (e.g., disrupted harmonics, altered energy distribution, atypical temporal structure) manifest in the time-frequency domain.

To ensure robust generalization assessment and prevent data leakage, we implement patient-level stratified splits. This means all recordings from a single patient are assigned exclusively to either the training set or the test set for each task, stratified by the diagnostic label. Further details on the experimental setup are provided in Section 4.

3.3. Multitask Model Architecture

To effectively model the complex acoustic manifestations associated with diverse vocal disorders, we design a multi-task dual-branch neural network. This architecture is designed to integrate information from both log-Mel spectrograms and MFCCs.

3.3.1. Modality-Specific Encoders

Input audio segments, preprocessed into the key representations described in Section 3.2, are fed into separate, specialized encoder backbones:

• •

Spectrogram Encoder ($f_{\text{spec}}$): An EfficientNet-B0 (tan2019efficientnet) architecture is employed to process the log-Mel spectrograms ($\mathbf{X}^{\text{spec}}$). To align with the standard 3-channel input expected by the ImageNet-pretrained model, the single-channel spectrogram is passed through an initial 2D convolutional layer that projects it to 3 channels before entering the EfficientNet backbone.

• •

MFCC Encoder ($f_{\text{mfcc}}$): A ResNet18 (he2016deep) architecture is used for the MFCC inputs ($\mathbf{X}^{\text{mfcc}}$). The first convolutional layer of the ResNet18 is modified to accept single-channel input, corresponding to the MFCC feature dimension over time.

The choice of these distinct encoders allows leveraging the specific strengths of EfficientNet for rich 2D spatial feature extraction from spectrograms and ResNet for capturing temporal patterns from MFCC sequences. Each encoder $f_{\text{modality}}$ outputs a modality-specific embedding:

[TABLE]

where $D_{\text{emb,mfcc}}=512$ (output dimension of ResNet18 average pooling layer) and $D_{\text{emb,spec}}=1280$ (output dimension of EfficientNet-B0 average pooling layer).

3.3.2. Feature Fusion and Shared Representation Layer

The modality-specific embeddings are subsequently concatenated to form a joint feature representation:

[TABLE]

In our case, $\mathbf{h}_{\text{fused}}\in\mathbb{R}^{1792}$. This fused representation is then processed by a shared transformation module, $g(\cdot)$, which consists of a linear projection layer (reducing dimensionality from 1792 to 512), followed by batch normalization, a LeakyReLU activation function ($\alpha=0.1$), and dropout. This module produces a task-agnostic shared latent vector $\mathbf{z}$:

[TABLE]

This shared representation layer aims to learn common underlying acoustic patterns relevant across multiple disorders.

3.3.3. Task-Specific Prediction Heads

For each of the $K$ diagnostic tasks (here $K=9$), a dedicated binary classification head, $h^{(k)}(\cdot)$, is appended to the shared representation layer. Each head is implemented as a two-layer Multi-Layer Perceptron (MLP):

• •

An initial linear layer transforming $\mathbf{z}$ from 512 to 128 dimensions, followed by batch normalization and LeakyReLU activation ($\alpha=0.1$).

• •

A final linear layer projecting the 128-dimensional representation to a single logit for binary classification.

• •

Dropout regularization is applied within each head.

Each head $h^{(k)}$ operates on the shared representation $\mathbf{z}$ to produce a task-specific prediction $\hat{y}^{(k)}$:

[TABLE]

where $\sigma(\cdot)$ is the sigmoid activation function, converting the logit to a probability. To address potential class imbalance inherent in clinical datasets, we employ a weighted binary cross-entropy (BCE) loss for each task:

[TABLE]

where $w_{1}^{(k)}$ and $w_{0}^{(k)}$ are weights assigned to the positive and negative classes for task $k$, respectively, typically set inversely proportional to class frequencies. The overall training objective is the sum of losses across all $K$ tasks:

[TABLE]

The training strategy and related details are provided in Section 4.1.

4. Experiments

4.1. Experimental Setup

This section details the specific configurations and choices made for training and evaluating our proposed models. The dataset, task definitions, and core model architecture have been described in Section 3.

Baselines and Model Configurations: To comprehensively assess the efficacy of our proposed multitask dual-modal framework (MARVEL), we compare it against several strong baselines:

• •

Single-Modality Deep Learning Models:

• –

$\text{EN}_{s}$** (EfficientNet-Spectrogram)**: An EfficientNet-B0 model trained solely on log-Mel spectrogram inputs.

• –

$\text{EN}_{m}$** (EfficientNet-MFCC)**: An EfficientNet-B0 model trained solely on MFCC inputs.

• –

$\text{RN}_{s}$** (ResNet-Spectrogram)**: A ResNet18 model trained solely on log-Mel spectrogram inputs.

• –

$\text{RN}_{m}$** (ResNet-MFCC)**: A ResNet18 model trained solely on MFCC inputs.

These single-modality models share the same encoder architecture as their respective branches in the MTL model but are trained in a single-task fashion for each of the nine disorders independently. Their outputs are then averaged or aggregated for comparison at the category level.

• •

MLP with Handcrafted Features (MLP): A shallow Multi-Layer Perceptron model trained on a comprehensive set of 131 traditional, handcrafted acoustic features (e.g., jitter, shimmer, HNR, formants). This baseline serves to evaluate the performance achievable with established acoustic biomarkers and provides context for our deep learning approaches.

Our proposed model, as detailed in Section 3.3, combines ResNet18 for MFCCs and EfficientNet-B0 for spectrograms, learning a shared representation across all nine diagnostic tasks simultaneously.

Training Details: All deep learning models (MTL and single-modality baselines) were trained using the AdamW optimizer with an initial learning rate of $1\times 10^{-4}$ and a weight decay of $1\times 10^{-5}$. A cosine annealing learning rate schedule was employed. We used a batch size of 108. For the MTL model, we implemented a multi-task balanced batch sampler, ensuring each mini-batch contained 6 positive and 6 negative samples per task (totaling $9\text{ tasks}\times 12\text{ samples/task}=108$ samples per batch). Sampling with replacement was used when insufficient unique samples were available for a given task within an epoch. All models were trained for a maximum of 40 epochs, with dropout regularization (rate of 0.3) applied to fully connected layers. Gradient clipping (norm 1.0) was used to ensure stability. The MLP baseline was trained using similar optimization hyperparameters.

Data Augmentation: To enhance model generalization, distinct data augmentation techniques were applied during training. For MFCC inputs, we appliep zero-mean Gaussian noise $\mathcal{N}(0,\sigma^{2})$, where $\sigma=0.01$. Further, we employed frequency masking, where 8 consecutive frequency bins were masked, and time masking, where 20 consecutive time frames were masked. For log-Mel spectrogram inputs, we utilized SpecAugment (park2019specaugment), which involved frequency masking (masking 15% of the frequency bins) and time masking (masking 15% of the time frames), in addition to the time warping procedure detailed in Section 3.3. These augmentations were applied stochastically exclusively during training.

Evaluation Metrics: The primary evaluation metric reported for all models and tasks is the Area Under the Receiver Operating Characteristic Curve (AUROC). AUROC provides a robust measure of classification performance, particularly well-suited for medical diagnostic tasks that may exhibit class imbalance. To ensure robustness and statistical significance, all experiments were conducted with five independent runs.

4.2. Results

In this section, we present a comprehensive evaluation of our proposed MARVEL  framework against various baselines across the nine defined diagnostic tasks. For completeness, a SHAP-based analysis of handcrafted features is included in Appendix A.

4.2.1. Comparison with Single-Task Single-Modality and MLP Baselines

Table 2 summarizes the AUROC performance (mean $\pm$ standard deviation) of our MARVEL  model compared to single-task, single-modality deep learning baselines ($\text{EN}_{m}$, $\text{EN}_{s}$, $\text{RN}_{m}$, $\text{RN}_{s}$) and the MLP model trained on handcrafted features. Performance is aggregated at the level of three high-level disorder categories: Respiratory, Voice, and Neurological, as well as an overall average across all tasks. The consistently low standard deviations (¡ 0.20 for most metrics) indicate stable training performance across different random initializations.

Each baseline model, including the MLP trained on handcrafted features, was trained independently for each task using a single type of acoustic representation—either MFCCs, spectrograms, or handcrafted features—without any task-sharing or modality fusion.

Our model consistently outperforms all single-task baselines across all aggregated disorder types, achieving the highest overall AUROC of $0.78\pm 0.14$. The model demonstrates a particularly strong discriminative capacity for neurological disorders ($0.89\pm 0.11$), followed by voice disorders ($0.76\pm 0.08$) and respiratory disorders ($0.74\pm 0.18$).

The advantages of the MARVEL  are most pronounced for voice disorders, where it achieves an AUROC of $0.76\pm 0.08$, a substantial improvement of $0.05$ AUROC points over the best-performing signle-task single-modal baseline in this category ($\text{EN}_{s}$ at $0.71\pm 0.09$). For neurological disorders, our model’s AUROC of $0.89\pm 0.11$ surpasses all alternatives, including the competitive MLP baseline ($0.80\pm 0.09$). While improvements for respiratory disorders are more modest, MARVEL  ($0.74\pm 0.18$) still offers consistent gains over single-modal approaches, which cluster around AUROCs of $0.70-0.73$.

These results underscore the effectiveness of our multi-task learning strategy. By leveraging shared representations across related disorders and integrating complementary information through dual-modal fusion (MFCCs and spectrograms), our model achieves superior generalization and discriminative power, particularly for conditions intrinsically linked to voice production mechanisms.

4.2.2. Task-Specific Performance Analysis

Figure 3 provides a detailed visualization of the MARVEL  task-level performance through Receiver Operating Characteristic (ROC) curves for each of the nine subtype disorders.Results are shown for the model with the best overall multi-task performance.These curves reveal distinct performance patterns across and within disease categories, offering insights into the model’s discriminative capabilities for specific conditions.

• •

Neurological Disorders (Blue ROC Curves): Exhibit exceptional performance overall. Alzheimer’s Disease/MCI detection achieves the highest discrimination (AUROC = $0.98$), indicating very strong separability. Parkinson’s Disease classification also demonstrates robust capability (AUROC = $0.81$).

• •

Respiratory Disorders (Green ROC Curves): Show more variable performance. Airway Stenosis classification yields excellent results (AUROC = $0.90$), while COPD detection is also strong (AUROC = $0.78$). Asthma, however, presents the most challenging task (AUROC = $0.56$), likely due to the subtlety or variability of its acoustic manifestations in the speech data.

• •

Voice Disorders (Purple ROC Curves): Generally demonstrate intermediate to strong performance. Vocal Fold Paralysis classification is a standout in this category (AUROC = $0.88$), indicating clear acoustic markers are learned by the model. Other voice disorders show moderate to good discrimination.

The variation in ROC curve shapes and AUROC values across tasks provides valuable information for potential clinical deployment strategies. High-performing tasks like AD/MCI and Airway Stenosis detection could support sensitive screening applications. Conversely, more challenging conditions like Asthma may require more conservative decision thresholds or integration with other diagnostic modalities (in particular breathing sounds), reflecting the underlying acoustic pathophysiology and guiding targeted clinical implementation.

4.2.3. Comparison with Self-Supervised Learning Baselines

To further contextualize the performance of our supervised MARVEL  framework, we compare it against contemporary self-supervised learning (SSL) models commonly used in speech processing. Table 3 presents a task-by-task AUROC comparison with Whisper-small (radford2023robust), Wav2Vec2-BERT (barrault2023seamless), and M-CTC-T (lugosch2022pseudo). For fair comparison, these SSL models were fine-tuned on our specific downstream tasks using the same training data splits and by their design only utilize spectrograms.

The results indicate that our model outperforms these SSL baselines on the majority of disorder subtypes (7 out of 9 tasks). Notably, MARVEL  achieves exceptional performance on Alzheimer’s Disease/MCI detection ($0.97\pm 0.01$) and Airway Stenosis classification ($0.89\pm 0.01$), surpassing the SSL models by a considerable margin.

While SSL models benefit from pretraining on large-scale, diverse speech corpora, they may not always capture the fine-grained, domain-specific acoustic biomarkers crucial for accurate medical diagnosis without extensive task-specific fine-tuning. In contrast, our MARVEL  framework, with its task-specific heads, dual-modal fusion tailored to relevant acoustic features, and supervised training on in-domain clinical data, learns more discriminative and clinically pertinent representations for voice-based disease classification. This highlights the current advantage of supervised, specialized architectures for these specific clinical tasks when sufficient labeled data is available.

4.2.4. Latent Space Visualization

To qualitatively assess the discriminative capacity of the representations learned by our model, we visualize the penultimate-layer embeddings (the 512-dimensional vector $\mathbf{z}$) using t-distributed Stochastic Neighbor Embedding (t-SNE) (van2008visualizing). Figure 4 presents these visualizations for two representative tasks: Alzheimer’s Disease/MCI (left panel) and Parkinson’s Disease (right panel), comparing embeddings from positive (disease) cases against negative (control) cases.

The t-SNE plot for AD/MCI reveals a clear and compact separation of positive and negative samples, with a distinct boundary between the two groups. This strong separation in the learned latent space aligns with the exceptionally high AUROC ($0.97$) observed for this task, suggesting that the model effectively captures stable and highly discriminative patterns for AD/MCI.

In contrast, the t-SNE visualization for Parkinson’s Disease shows a slightly more diffuse cluster boundary. While distinct clusters for positive and negative cases are still evident, there is greater intermingling compared to the AD/MCI plot. This suggests that while the model learns condition-specific signals for Parkinson’s Disease, these signals might exhibit greater variability or subtlety, consistent with the more moderate AUROC of $0.81$ for this task.

Overall, these t-SNE visualizations provide compelling visual evidence that our dual-modal architecture successfully encodes task-relevant acoustic features, learning a latent space that exhibits strong separability for clinically relevant conditions, thereby corroborating the quantitative performance metrics.

4.3. Ablation Study and Component Analysis

To dissect the contributions of our framework’s components, we conducted an ablation study comparing our dual-modal multi-task model against single-modality models ($\text{EN}_{m}$, $\text{EN}_{s}$, $\text{RN}_{m}$, $\text{RN}_{s}$) and an MLP baseline across all nine tasks. Detailed AUROC scores are provided in Table 4.

4.3.1. Impact of Multi-Task Learning (MTL)

MARVEL  framework demonstrates superior performance by outperforming single-task baselines on 7 of 9 tasks. The benefits are most notable for tasks like Airway Stenosis (AUROC $0.89\pm 0.01$ vs. best single-modality $0.85\pm 0.04$), Vocal Fold Paralysis ($0.87\pm 0.01$ vs. $0.77\pm 0.02$), and Alzheimer’s Disease/MCI ($0.97\pm 0.01$ vs. $0.86\pm 0.01$ for MLP). These improvements, also seen in moderately resourced tasks (e.g., Parkinson’s, laryngeal cancer), affirm that shared supervision in MARVELfosters robust representation learning, particularly with limited task-specific labeled data.

4.3.2. Complementarity of Acoustic Modalities (MFCCs vs. Spectrograms)

Analysis of single-modality performance reveals task-specific strengths: MFCC-based models often excel for neurological disorders due to sensitivity to temporal and prosodic features, while respiratory and structural voice disorders show mixed preferences. Critically, our dual-modal model surpasses the best single-modality configuration on 8 of 9 tasks, with AUROC gains ranging up to $0.19$ points (e.g., Laryngeal Cancer). This underscores that MFCCs (capturing spectral envelope and dynamics) and spectrograms (rich time-frequency details) provide complementary acoustic information, and their fusion yields more discriminative representations than either modality alone, with Spasmodic Dysphonia being a minor exception.

4.3.3. Influence of Task-Intrinsic Difficulty on Performance

A consistent performance hierarchy across tasks is observed in all models. High-performing tasks like Alzheimer’s Disease/MCI (AUROC $0.97$) and Airway Stenosis ($0.89$) suggest distinct acoustic biomarkers. Moderately-performing tasks include Parkinson’s Disease ($0.81$). Asthma remains a low-performing task ($0.54$), likely due to subtle or variable acoustic cues, or data ambiguities, rather than limitations of the model itself. As an episodic condition, participants may have been asymptomatic during recording, reducing the presence of characteristic markers. The absence of healthy controls further complicates discrimination by requiring differentiation from other respiratory disorders with overlapping acoustic profiles. These factors suggest limited diagnostic utility of voice for conditions with intermittent symptoms or similar acoustic phenotypes. This indicates that task-intrinsic difficulty is a primary factor constraining performance for certain conditions.

4.4. Correlation Analysis between Model Embeddings and Acoustic Biomarkers

To further examine the relationship between learned representations and established acoustic biomarkers within our multi-task framework, we conducted a post hoc correlation analysis between the learned deep embeddings and traditional handcrafted acoustic features. Specifically, we used the 128-dimensional embeddings from the final fused layer of the MARVEL  model on the test set and computed Pearson correlation coefficients with all 131 handcrafted acoustic features extracted from each audio sample. These features, derived using openSMILE and Praat toolkits, include a broad spectrum of prosodic, spectral, and voice quality descriptors such as F0_sma_stddev (pitch variability), logHNR_sma3nz_stddevNorm (harmonic-to-noise ratio), and slopeUV0-500_sma3nz_amean (spectral slope in the low-frequency band).

Figure 5 presents the top-5 acoustic features most correlated with the learned embeddings for each diagnostic task. These results reveal meaningful task-specific associations between clinically interpretable acoustic attributes and internal model representations, suggesting that MARVEL  implicitly captures patterns consistent with domain knowledge—even without explicit access to engineered features.

In several tasks, we observed strong correlations between learned deep embeddings and clinically meaningful handcrafted acoustic features. In Laryngeal cancer, the highest correlations were found with HNRdBACF_sma3nz_amean ($r=0.809$) and the other feature F3frequency_sma3nz_amean ($r=0.782$), which reflect the harmonic-to-noise ratio and third formant frequency, critical indicators of spectral stability and vocal tract configuration often disrupted in malignant laryngeal conditions. For Benign Lesions in the Vocal Cord task, features such as logRelF0-H1-H2_sma3nz_amean showed high correlation values ($r=0.748$, $0.732$ and $0.721$ in variants, aligned with irregularities at the glottal source commonly associated with benign structural lesions. In COPD, MFCC-based characteristics, such as mfcc4_sma3_amean ($r=0.713$), exhibited strong correlations, suggesting that the model captures spectral envelope patterns potentially linked to altered airflow and resonance seen in chronic pulmonary conditions. For Vocal Fold Paralysis, features like loudness_sma3_percentile20.0 ($r=0.602$) and std_intensity_db ($r=0.573$) were most correlated, corresponding to reduced vocal intensity and instability—hallmarks of glottal incompetence. Even in Parkinson’s disease, where shows moderate correlations, consistent alignment with the features such as slopeUV0-500_sma3nz_amean variants ($r\approx 0.46$–$0.51$) supports the model’s sensitivity to subtle prosodic deficits such as reduced pitch modulation and breathiness, which are typical in Parkinsonian speech.

Rather than quantifying feature importance, this correlation analysis assesses the extent to which the model’s learned representations reflect clinically grounded acoustic attributes. The observed task-specific alignments reinforce the clinical plausibility of the internal embeddings.

Importantly, while a lightweight MLP trained on handcrafted acoustic features demonstrates meaningful discriminative performance and captures clinically relevant voice characteristics, our MARVEL  model consistently outperforms it. This performance gap suggests that MARVEL  not only encodes similar diagnostic cues but also learns more comprehensive and nuanced vocal patterns directly from derived acoustic representations—highlighting the value of end-to-end learning beyond manual feature engineering.

We further provide SHAP-based analysis in Appendix A, which confirms that many of the highly correlated acoustic features also contribute to disorder detection. Together, these findings suggest that MARVEL  captures clinically meaningful voice patterns while offering richer and more expressive representations than traditional approaches.

5. Conclusion

In this work, we introduced MARVEL, a multi-task, dual-modal deep learning framework that effectively detects diverse neurological, respiratory, and voice disorders by operating on speech derived MFCC and spectrogram features. Evaluated on the large-scale Bridge2AI-Voice dataset, MARVEL  demonstrated superior performance over single-task, single-modal and self-supervised baselines, with its dual-modal fusion yielding consistent 5–19% performance gains. A post hoc analysis revealed that the model’s learned embeddings correlate with clinically meaningful handcrafted acoustic features, offering interpretability into the vocal patterns MARVEL  implicitly captures. Our findings underscore the significant potential of scalable, non-invasive voice-based diagnostics, establishing a strong foundation for clinical applications. Future work will focus on integrating self-supervised pre-training to leverage unlabeled voice data and incorporating uncertainty quantification to enhance clinical decision-making, ultimately paving the way for deployment-ready AI systems in healthcare practice.

Acknowledgments

This work was supported by the AiNed Fellowship Grant awarded to A.S. We acknowledge the use of the Dutch National Supercomputer Snellius for essential computational tasks.

Appendix

Appendix A SHAP-Based Feature Attribution for Acoustic Biomarkers

To complement our primary deep learning framework and contextualize the findings from Section 4.4, which revealed correlations between learned embeddings and handcrafted acoustic features, we conducted an auxiliary SHAP-based analysis. This involved training a lightweight Multi-Layer Perceptron (MLP) model on the same set of handcrafted acoustic features used in the correlation study.

The objectives of this analysis were twofold: (1) to independently validate whether traditional, interpretable acoustic features are predictive of voice-related disorders, and (2) to identify which specific acoustic descriptors are most influential for each classification task using SHAP (SHapley Additive Explanations) (lundberg2017unified).

Together, this SHAP analysis offers a complementary interpretability perspective—highlighting which features are critical for performance when using only handcrafted descriptors, and reinforcing the clinical plausibility of the vocal patterns captured by MARVEL’s learned representations.

A.1. Handcrafted Acoustic Feature Set and MLP Performance

The handcrafted acoustic feature set in Bridge2AI-Voice dataset was compiled using established open-source signal processing toolkits, like openSMILE and Praat. This extensive pipeline extracts a diverse array of descriptors, encompassing spectral characteristics (e.g., spectral flux, roll-off), prosodic patterns (e.g., pitch-related features, speaking rate), voice quality measures (e.g., jitter, shimmer, Harmonic-to-Noise Ratio), and temporal dynamics. Each feature represents a global summary statistic computed over an entire speech recording, providing a structured, interpretable representation of vocal biomarkers.

As previously reported in Table 2 (Section 4.2.1), the MLP classifier trained solely on these handcrafted features demonstrated commendable performance across the high-level disorder categories. It achieved AUROC scores of $0.80\pm 0.09$ for neurological disorders, $0.67\pm 0.09$ for voice disorders, and $0.71\pm 0.15$ for respiratory disorders. These results confirm that traditional, structured acoustic descriptors indeed capture discriminative patterns relevant to various health conditions, particularly for neurodegenerative disorders where subtle changes in motor control often manifest in measurable vocal characteristics, even without leveraging complex deep learning architectures or raw spectral inputs.

A.2. Identifying Salient Acoustic Features with SHAP

To understand which specific acoustic features drive the MLP’s classification decisions for different disorders, we applied SHAP analysis. Figure 6 illustrates the top-five most impactful features for each of the nine disease subtypes, ranked by their mean absolute SHAP value. This value indicates the average magnitude of a feature’s contribution to the model’s output across all samples. Each bar represents the relative discriminative importance of the feature for the corresponding disorder classification.

The analysis reveals distinct feature importance profiles across disorder categories, many of which align with known clinical voice characteristics. For neurological disorders such as Parkinson’s Disease and AD/MCI, features related to pitch stability and articulatory precision, including slopeUV0-500 (reflecting pitch trajectory in unvoiced segments) and alphaRatioUV (related to spectral tilt), were prominent. The importance of mean_F2_loc (average second formant location) further suggests changes in vocal tract configuration, aligning with known deficits in motor control and prosodic modulation common in these conditions. In the case of respiratory disorders like Airway Stenosis and COPD, features such as spectral_gravity (spectral center of mass), MFCC-related statistics (e.g., mfcc2V_sma3nz), and intensity measures (e.g., mean_intensity_db) proved significant. These often reflect changes in subglottal pressure, airflow turbulence, and overall vocal effort, consistent with impaired pulmonary function. Similarly, voice disorders including Vocal Fold Paralysis and Spasmodic Dysphonia were primarily characterized by cepstral features (e.g., cepstral_peak_prominence) and other MFCC-derived metrics. These descriptors are sensitive to irregularities in glottal closure and vocal fold vibration patterns, which are hallmarks of laryngeal pathologies. Conversely, for conditions like Asthma where acoustic cues may be less consistently strong, SHAP values were generally more diffuse and lower in magnitude.

These findings collectively demonstrate that an interpretable model based on handcrafted acoustic features can not only achieve reasonable classification performance but also highlight pathophysiologically relevant vocal dimensions. This provides valuable context, suggesting that the acoustic cues exploited by our more complex model are likely rooted in these fundamental, clinically meaningful vocal characteristics.

Clinical Interpretation of Feature Relevance. Notably, the acoustic features identified as salient by SHAP correspond to well-established physiological correlates. Perturbations in jitter and shimmer reflect vocal fold instability and irregular phonation, commonly associated with laryngeal or neurogenic dysphonia (lopes2017accuracy). Another acoustic feature Spectral_gravity, which measures energy distribution across frequencies, is indicative of airflow limitation and has been linked to compromised respiratory mechanics (taylor2020age). Cepstral features such as cepstral_peak_prominence relate to harmonic structure and are sensitive to glottal closure behavior, which is relevant in vocal fold paralysis and tremor (balasubramanium2011cepstral). Similarly, formant-based features like mean_F2_loc capture shifts in vocal tract resonance and articulatory posture, which are often impaired in neurodegenerative disorders (mirarchi2017signal). Collectively, these associations reinforce the biological validity of acoustic biomarkers and highlight their value as non-invasive indicators of clinical conditions, thereby supporting the foundational premise of voice-based disease modeling adopted in our primary framework.

A.3. Universality versus Specificity of Acoustic Biomarkers

To investigate whether certain acoustic biomarkers are broadly indicative of vocal pathology (universal) versus specific to certain conditions, we analyzed the recurrence of the top-5 SHAP-ranked features across the nine disease-specific MLP models. Table 5 summarizes key features appearing in the top-5 list for multiple disorders, along with their average SHAP value (computed only for diseases where they ranked in the top-5) and associated conditions.

Several patterns emerge from this cross-condition analysis. Some features demonstrate broad relevance across related disorder families. For instance, slopeUV0-500_sma3nz_amean (average slope of unvoiced segments) shows high importance across three neurologically-related conditions (AD/MCI, Spasmodic Dysphonia, Parkinson’s), suggesting its utility in capturing laryngeal motor control impairments common to these disorders. Similarly, mfcc2V_sma3nz_amean (second MFCC variant) is prominent across three structural/functional voice disorders (Airway Stenosis, Spasmodic Dysphonia, Vocal Fold Paralysis), likely reflecting alterations in vocal tract resonance due to anatomical or phonatory changes. Features like spectral_gravity and loudness_sma3_percentile80.0 also appear across three respiratory-related conditions, indicating systematic effects of respiratory limitations on sound energy and loudness.

This analysis reveals a hierarchy of acoustic biomarkers. Some features, like those related to unvoiced segment slopes or basic spectral balance, appear to be somewhat universal indicators of vocal system disruption across related disorder families. Others, while still shared between a couple of conditions, offer more targeted diagnostic information. This observation supports a layered approach to voice biomarker discovery, where both broadly indicative and highly specific features contribute to a comprehensive understanding of vocal health. The insights derived from these interpretable models reinforce the premise that speech contains rich, multi-faceted information about underlying health states, which more complex models like our MARVEL  framework can potentially exploit more effectively.