Multilingual Audio-Visual Smartphone Dataset and Evaluation

Smartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an...

Full description

Saved in:
Bibliographic Details
Published inIEEE access Vol. 9; pp. 153240 - 153257
Main Authors Mandalapu, Hareesh, Reddy, P. N. Aravinda, Ramachandra, Raghavendra, Rao, Krothapalli Sreenivasa, Mitra, Pabitra, Prasanna, S. R. Mahadeva, Busch, Christoph
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Algorithms
Audio equipment
audio-visual speaker recognition
Biometrics
Biometrics (access control)
Datasets
Face recognition
multilingual
Performance evaluation
presentation attack detection
Sensors
Smart phones
Smartphone biometrics
Smartphones
Speech recognition
Verification
Visualization
Online AccessGet full text

Cover

More Information
Summary:Smartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an audio-visual smartphone dataset captured in five different recent smartphones. This new dataset contains 103 subjects captured in three different sessions considering the different real-world scenarios. Three different languages are acquired in this dataset to include the problem of language dependency of the speaker recognition systems. These unique characteristics of this dataset will pave the way to implement novel state-of-the-art unimodal or audio-visual speaker recognition systems. We also report the performance of the bench-marked biometric verification systems on our dataset. The robustness of biometric algorithms is evaluated towards multiple dependencies like signal noise, device, language and presentation attacks like replay and synthesized signals with extensive experiments. The obtained results raised many concerns about the generalization properties of state-of-the-art biometrics methods in smartphones.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2021.3125485