The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance

• Published in: IEEE/ACM transactions on audio, speech, and language processing Vol. 31; pp. 813 - 825
• Main Authors: Zhang, Lin, Wang, Xin, Cooper, Erica, Evans, Nicholas, Yamagishi, Junichi
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Anti-spoofing; Computer Science; countermeasure; deepfake; Engineering Sciences; Error detection; Feature extraction; Forgery; Internet; Labels; PartialSpoof; Segments; self-supervised learning; Speech processing; Speech recognition; Splicing; spoof localization; Spoofing; PartialSpoof; spoof localization; Anti-spoofing; countermeasure; selfsupervised learning; deepfake
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2022.3233236

Automatic speaker verification is susceptible to various manipulations and spoofing, such as text-to-speech synthesis, voice conversion, replay, tampering, adversarial attacks, and so on. We consider a new spoofing scenario called "Partial Spoof" (PS) in which synthesized or transformed speech segments are embedded into a bona fide utterance. While existing countermeasures (CMs) can detect fully spoofed utterances, there is a need for their adaptation or extension to the PS scenario. We propose various improvements to construct a significantly more accurate CM that can detect and locate short-generated spoofed speech segments at finer temporal resolutions. First, we introduce newly developed self-supervised pre-trained models as enhanced feature extractors. Second, we extend our PartialSpoof database by adding segment labels for various temporal resolutions. Since the short spoofed speech segments to be embedded by attackers are of variable length, six different temporal resolutions are considered, ranging from as short as 20 ms to as large as 640 ms. Third, we propose a new CM that enables the simultaneous use of the segment-level labels at different temporal resolutions as well as utterance-level labels to execute utterance- and segment-level detection at the same time. We also show that the proposed CM is capable of detecting spoofing at the utterance level with low error rates in the PS scenario as well as in a related logical access (LA) scenario. The equal error rates of utterance-level detection on the PartialSpoof database and ASVspoof 2019 LA database were 0.77 and 0.90%, respectively.