End-to-end speech diarization via iterative speaker embedding

• Main Authors: Grangier, David, Teboul, Oliver, Zeghidour, Neil
• Format: Patent
• Language: English
• Published: 30.01.2024
• Subjects: ACOUSTICS; CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; MUSICAL INSTRUMENTS; PHYSICS; SPEECH ANALYSIS OR SYNTHESIS; SPEECH OR AUDIO CODING OR DECODING; SPEECH OR VOICE PROCESSING; SPEECH RECOGNITION

A method includes receiving an input audio signal corresponding to utterances spoken by multiple speakers. The method also includes encoding the input audio signal into a sequence of T temporal embeddings. During each of a plurality of iterations each corresponding to a respective speaker of the multiple speakers, the method includes selecting a respective speaker embedding for the respective speaker by determining a probability that the corresponding temporal embedding includes a presence of voice activity by a single new speaker for which a speaker embedding was not previously selected during a previous iteration and selecting the respective speaker embedding for the respective speaker as the temporal embedding. The method also includes, at each time step, predicting a respective voice activity indicator for each respective speaker of the multiple speakers based on the respective speaker embeddings selected during the plurality of iterations and the temporal embedding.