Blind Speech Separation and Dereverberation using neural beamforming

• Published in: Speech communication Vol. 140; pp. 29 - 41
• Main Authors: Pfeifenberger, Lukas, Pernkopf, Franz
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.05.2022; Elsevier Science Ltd
• Subjects: Beamforming; Dereverberation; Error analysis; Multi-channel speaker separation; Neural networks; Separation; Speaker identification; Speech; Triplet mining; Voice recognition; Beamforming; Triplet mining; Speaker identification; Multi-channel speaker separation; Dereverberation
• ISSN: 0167-6393; 1872-7182
• DOI: 10.1016/j.specom.2022.03.004

In this paper, we present the Blind Speech Separation and Dereverberation (BSSD) network, which performs simultaneous speaker separation, dereverberation and speaker identification in a single neural network. Speaker separation is guided by a set of predefined spatial cues. Dereverberation is performed by using neural beamforming, and speaker identification is aided by embedding vectors and triplet mining. We introduce a frequency-domain model which uses complex-valued neural networks, and a time-domain variant which performs beamforming in latent space. Further, we propose a block-online mode to process longer audio recordings, as they occur in meeting scenarios. We evaluate our system in terms of Scale Independent Signal to Distortion Ratio (SI-SDR), Word Error Rate (WER) and Equal Error Rate (EER). •A blind speaker separation architecture, using both linear signal processing and artificial neural networks.•GCC-PHAT is used to iteratively localize an unknown number speaker positions.•A DNN is used to separate, dereverberate and identify each speaker from a given mixture.•Both offline and online operation modes are evaluated in the experiments.•Performance, computational efficiency, SDI and WER are analyzed and compared against state-of-the-art methods.