Improving Text-Independent Forced Alignment to Support Speech-Language Pathologists with Phonetic Transcription

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 24; p. 9650
• Main Authors: Li, Ying, Wohlan, Bryce Johannas, Pham, Duc-Son, Chan, Kit Yan, Ward, Roslyn, Hennessey, Neville, Tan, Tele
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 01.12.2023; MDPI
• Subjects: Accuracy; Acoustics; Algorithms; Alignment; Classification; Datasets; Diagnostic tests; Disease susceptibility; forced alignment; Language; Medical research; Medicine, Experimental; Neural networks; phoneme segmentation; Phonemes; Phonetic transcription; Phonetics; phonological disorders; Segmentation; Speech disorders; Speech perception; speech sound disorders; Speech therapists; speech therapy; Speech-language pathologists; Speech-language pathology; Voice recognition; wav2vec 2.0; phonological disorders; phoneme segmentation; speech sound disorders; speech therapy; forced alignment; wav2vec 2.0
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23249650

: Phonetic transcription is crucial in diagnosing speech sound disorders (SSDs) but is susceptible to transcriber experience and perceptual bias. Current forced alignment (FA) tools, which annotate audio files to determine spoken content and its placement, often require manual transcription, limiting their effectiveness. : We introduce a novel, text-independent forced alignment model that autonomously recognises individual phonemes and their boundaries, addressing these limitations. Our approach leverages an advanced, pre-trained wav2vec 2.0 model to segment speech into tokens and recognise them automatically. To accurately identify phoneme boundaries, we utilise an unsupervised segmentation tool, UnsupSeg. Labelling of segments employs nearest-neighbour classification with wav2vec 2.0 labels, before connectionist temporal classification (CTC) collapse, determining class labels based on maximum overlap. Additional post-processing, including overfitting cleaning and voice activity detection, is implemented to enhance segmentation. : We benchmarked our model against existing methods using the TIMIT dataset for normal speakers and, for the first time, evaluated its performance on the TORGO dataset containing SSD speakers. Our model demonstrated competitive performance, achieving a harmonic mean score of 76.88% on TIMIT and 70.31% on TORGO. : This research presents a significant advancement in the assessment and diagnosis of SSDs, offering a more objective and less biased approach than traditional methods. Our model's effectiveness, particularly with SSD speakers, opens new avenues for research and clinical application in speech pathology.