The development of robust automated systems for assessing speech intelligibility levels in patients with dysarthria is currently hindered by the scarcity of labeled pathological datasets. This study proposes a novel framework addressing this limitation by integrating self-supervised learning with generative data augmentation. We employ Wav2Vec 2.0, a pre-trained self-supervised speech model, to extract and analyze contextualized representations, aligned with both grapheme and phoneme targets, systematically identifying which linguistic units serve as the most effective biomarkers for binary intelligibility classification. To address data scarcity, we employ StyleTTS-2, a diffusion-based generative model, to generate synthetic dysarthric speech samples conditioned on reference audio from the TORGO dataset and text prompts from the UA-Speech dataset. Experimental results demonstrate that contextualized representations yield superior classification accuracy compared to approaches operating on full word- or utterance-level audio. Significantly higher performance is observed on units most degraded by dysarthric impairment, validating the proposed fine-grained approach. Furthermore, synthetic data augmentation proved to enhance model robustness, effectively mitigating data scarcity issues.

Assessing Dysarthric Speech Intelligibility via Generative Augmentation and Self-supervised Linguistic Representations / Lillini, D., Aironi, C., Migliorelli, L., Gabrielli, L., Squartini, S.. - 3206:(2027), pp. 193-207. (6th International Conference on Applied Intelligence and Informatics, AII 2026 gbr 2026) [10.1007/978-3-032-42348-1_14].

Assessing Dysarthric Speech Intelligibility via Generative Augmentation and Self-supervised Linguistic Representations

Lillini D.;Aironi C.;Gabrielli L.;Squartini S.
2027-01-01

Abstract

The development of robust automated systems for assessing speech intelligibility levels in patients with dysarthria is currently hindered by the scarcity of labeled pathological datasets. This study proposes a novel framework addressing this limitation by integrating self-supervised learning with generative data augmentation. We employ Wav2Vec 2.0, a pre-trained self-supervised speech model, to extract and analyze contextualized representations, aligned with both grapheme and phoneme targets, systematically identifying which linguistic units serve as the most effective biomarkers for binary intelligibility classification. To address data scarcity, we employ StyleTTS-2, a diffusion-based generative model, to generate synthetic dysarthric speech samples conditioned on reference audio from the TORGO dataset and text prompts from the UA-Speech dataset. Experimental results demonstrate that contextualized representations yield superior classification accuracy compared to approaches operating on full word- or utterance-level audio. Significantly higher performance is observed on units most degraded by dysarthric impairment, validating the proposed fine-grained approach. Furthermore, synthetic data augmentation proved to enhance model robustness, effectively mitigating data scarcity issues.
2027
9783032423474
9783032423481
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11566/366854
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact