Method and apparatus for real-time sound enhancement

• Main Authors: Syed Samin Ishtiaq, Carlos Purves, Abhinav Mehrotra, Alberto Gil C. P. Ramos, Sourav Bhattacharya, Ravichander Vipperla, Nicholas Donald Atkins Lane
• Format: Patent
• Language: English
• Published: 28.09.2022
• Subjects: ACOUSTICS; CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; MUSICAL INSTRUMENTS; PHYSICS; SPEECH ANALYSIS OR SYNTHESIS; SPEECH OR AUDIO CODING OR DECODING; SPEECH OR VOICE PROCESSING; SPEECH RECOGNITION

A system, computer-implemented method and apparatus for training a machine learning, ML, model to perform sound enhancement for a target user in real-time, and to a method and apparatus for using the trained ML model to perform sound enhancement of audio signals in real-time. the invention is suitable for implementation on resource constrained devices that capture audio signals, such as smartphones and Internet of Things devices. The method comprises: obtaining a noisy audio signal comprising speech of a target user and noise; determining whether a speaker embedding vector for the target user exists; and using neural networks of the trained ML model, to remove the noise from the noisy audio signal while maintaining the speech of the target user by switching the trained ML model to perform personalised or non-personalised noise removal depending on whether the speaker embedding vector exists.