A comprehensive multimodal dataset for contactless lip reading and acoustic analysis
Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, w...
Saved in:
| Published in | Scientific data Vol. 10; no. 1; pp. 895 - 17 |
|---|---|
| Main Authors | , , , , , , , , , , |
| Format | Journal Article |
| Language | English |
| Published |
London
Nature Publishing Group UK
13.12.2023
Nature Publishing Group Nature Portfolio |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5
GHz
Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77
GHz
frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject’s lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition. |
|---|---|
| AbstractList | Abstract Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject’s lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition. Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject's lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition. Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject’s lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition. Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject's lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition.Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our dataset paper aims to facilitate the enhancement and restoration of speech information from diverse data sources for speakers. In this paper, we introduce a novel multimodal dataset based on Radio Frequency, visual, text, audio, laser and lip landmark information, also called RVTALL. Specifically, the dataset consists of 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77 GHz frequency modulated continuous wave (FMCW) data from millimeter wave (mmWave) radar, visual and audio information, lip landmarks and laser data, offering a unique multimodal approach to speech recognition research. Meanwhile, a depth camera is adopted to record the landmarks of the subject's lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words, and 16 sentences. The dataset has been validated and has potential for the investigation of lip reading and multimodal speech recognition. |
| ArticleNumber | 895 |
| Author | Chen, Zikang Cooper, Jonathan Chetty, Kevin Faccio, Daniele Imran, Muhammad Tang, Chong Abbasi, Qammer H. Ge, Yao Li, Haobo Wang, Jingyan Li, Wenda |
| Author_xml | – sequence: 1 givenname: Yao orcidid: 0000-0001-6314-3891 surname: Ge fullname: Ge, Yao organization: James Watt School of Engineering, University of Glasgow – sequence: 2 givenname: Chong surname: Tang fullname: Tang, Chong organization: James Watt School of Engineering, University of Glasgow, Department of Security and Crime Science, University College London – sequence: 3 givenname: Haobo surname: Li fullname: Li, Haobo organization: School of Physics & Astronomy, University of Glasgow – sequence: 4 givenname: Zikang surname: Chen fullname: Chen, Zikang organization: James Watt School of Engineering, University of Glasgow – sequence: 5 givenname: Jingyan surname: Wang fullname: Wang, Jingyan organization: James Watt School of Engineering, University of Glasgow – sequence: 6 givenname: Wenda orcidid: 0000-0001-6617-9136 surname: Li fullname: Li, Wenda organization: School of Science and Engineering, University of Dundee – sequence: 7 givenname: Jonathan surname: Cooper fullname: Cooper, Jonathan organization: James Watt School of Engineering, University of Glasgow – sequence: 8 givenname: Kevin surname: Chetty fullname: Chetty, Kevin organization: Department of Security and Crime Science, University College London – sequence: 9 givenname: Daniele orcidid: 0000-0001-8397-334X surname: Faccio fullname: Faccio, Daniele organization: School of Physics & Astronomy, University of Glasgow – sequence: 10 givenname: Muhammad orcidid: 0000-0003-4743-9136 surname: Imran fullname: Imran, Muhammad organization: James Watt School of Engineering, University of Glasgow – sequence: 11 givenname: Qammer H. surname: Abbasi fullname: Abbasi, Qammer H. email: qammer.abbasi@glasgow.ac.uk organization: James Watt School of Engineering, University of Glasgow |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/38092796$$D View this record in MEDLINE/PubMed |
| BookMark | eNp9Uk1vFSEUJabG1to_4MJM4sbNKFxgGFamafxo0sRNXRMGmFdeGHjCTJv-e3lvam1duLgBLuccTi7nNTqKKTqE3hL8kWDafyqMcClaDLSWkLS9e4FOAHNoGevo0ZP9MTorZYsxJpRhLvArdEx7LCupO0HX541J0y67GxeLv3XNtITZT8nq0Fg96-LmZky5guKszRxcKU3wuyY7bX3cNDraRpu0lNmbetDhvvjyBr0cdSju7GE9RT-_frm--N5e_fh2eXF-1RomYW5J37ne9MJaKxkQzEdrRiaosQADFlQy0tW-Bg5CV78DZ9CBGGHEo-NC0FN0uerapLdql_2k871K2qtDI-WN0rkaC045aztN5AACKBsAayk4AYqrNun7gVetz6vWbhkmZ42Lc9bhmejzm-hv1CbdKoIFkdD1VeHDg0JOvxZXZjX5YlwIOro6IAUSg-RMAKnQ9_9At2nJdXoHFNlXvxd899TSo5c_v1cBsAJMTqVkNz5CCFb7lKg1JaqmRB1Sou4qia6kUsFx4_Lft__D-g26074j |
| Cites_doi | 10.1109/JIOT.2022.3217968 10.1038/s41467-022-32231-1 10.1007/s10579-020-09503-7 10.1121/1.2229005 10.3390/app11219870 10.1109/TASLP.2023.3250846 10.3390/s23010330 10.1364/BOE.451416 10.1007/s10579-008-9076-6 10.1016/j.neuroimage.2019.116283 10.1109/ACCESS.2020.3026579 10.6084/m9.figshare.c.6942846.v1 10.3390/s22020649 10.1109/JSEN.2019.2946095 10.1109/TMTT.2014.2320464 10.1109/JSEN.2018.2877043 10.1109/COMST.2022.3145856 10.3390/s21103465 10.1121/1.397959 10.1371/journal.pone.0196391 10.1016/j.procs.2021.12.222 10.1145/3277883.3277884 10.1145/3539490.3539599 10.1016/j.specom.2007.02.006 10.1109/TNSRE.2023.3266299 10.1145/3447993.3448626 10.1109/SLT48900.2021.9383619 10.1145/3397321 10.1145/3485730.3485945 10.1109/CVPR46437.2021.00500 |
| ContentType | Journal Article |
| Copyright | The Author(s) 2023 2023. The Author(s). The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: The Author(s) 2023 – notice: 2023. The Author(s). – notice: The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | C6C AAYXX CITATION NPM 3V. 7X7 7XB 88E 8FE 8FH 8FI 8FJ 8FK ABUWG AFKRA AZQEC BBNVY BENPR BHPHI CCPQU DWQXO FYUFA GHDGH GNUQQ HCIFZ K9. LK8 M0S M1P M7P PHGZM PHGZT PIMPY PJZUB PKEHL PPXIY PQEST PQGLB PQQKQ PQUKI PRINS 7X8 5PM DOA |
| DOI | 10.1038/s41597-023-02793-w |
| DatabaseName | Springer Nature OA Free Journals CrossRef PubMed ProQuest Central (Corporate) ProQuest Health & Medical Collection (NC LIVE) ProQuest Central (purchase pre-March 2016) Medical Database (Alumni Edition) ProQuest SciTech Collection ProQuest Natural Science Collection Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials Biological Science Collection ProQuest Central Natural Science Collection ProQuest One Community College ProQuest Central Korea Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student SciTech Premium Collection (via ProQuest) ProQuest Health & Medical Complete (Alumni) ProQuest Biological Science Collection ProQuest Health & Medical Collection Medical Database Biological Science Database ProQuest Central Premium ProQuest One Academic Publicly Available Content Database ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest One Health & Nursing ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China MEDLINE - Academic PubMed Central (Full Participant titles) DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef PubMed Publicly Available Content Database ProQuest Central Student ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest One Health & Nursing ProQuest Natural Science Collection ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Health & Medical Research Collection Health Research Premium Collection Health and Medicine Complete (Alumni Edition) Natural Science Collection ProQuest Central Korea Health & Medical Research Collection Biological Science Collection ProQuest Central (New) ProQuest Medical Library (Alumni) ProQuest Biological Science Collection ProQuest One Academic Eastern Edition ProQuest Hospital Collection Health Research Premium Collection (Alumni) Biological Science Database ProQuest SciTech Collection ProQuest Hospital Collection (Alumni) ProQuest Health & Medical Complete ProQuest Medical Library ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) MEDLINE - Academic |
| DatabaseTitleList | PubMed Publicly Available Content Database CrossRef MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: C6C name: Springer Nature OA Free Journals url: http://www.springeropen.com/ sourceTypes: Publisher – sequence: 2 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 3 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 4 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Sciences (General) |
| EISSN | 2052-4463 |
| EndPage | 17 |
| ExternalDocumentID | oai_doaj_org_article_edd6a19b27234b20a9751230252188b5 PMC10719268 38092796 10_1038_s41597_023_02793_w |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: RCUK | Engineering and Physical Sciences Research Council (EPSRC) grantid: EP/T021063/1; EP/W003228/1; EP/T021020/1 funderid: https://doi.org/10.13039/501100000266 – fundername: Royal Society of Edinburgh (RSE) grantid: SAPHIRE funderid: https://doi.org/10.13039/501100000332 – fundername: RCUK | Engineering and Physical Sciences Research Council (EPSRC) grantid: EP/T021063/1 – fundername: RCUK | Engineering and Physical Sciences Research Council (EPSRC) grantid: EP/T021020/1 – fundername: RCUK | Engineering and Physical Sciences Research Council (EPSRC) grantid: EP/W003228/1 – fundername: Royal Society of Edinburgh (RSE) grantid: SAPHIRE |
| GroupedDBID | 0R~ 3V. 53G 5VS 7X7 88E 8FE 8FH 8FI 8FJ AAJSJ ABUWG ACGFS ACSFO ACSMW ADBBV ADRAZ AFKRA AGHDO AJTQC ALIPV ALMA_UNASSIGNED_HOLDINGS AOIJS BBNVY BCNDV BENPR BHPHI BPHCQ BVXVI C6C CCPQU DIK EBLON EBS EJD FYUFA GROUPED_DOAJ HCIFZ HMCUK HYE KQ8 LK8 M1P M48 M7P M~E NAO OK1 PGMZT PIMPY PQQKQ PROAC PSQYO RNT RNTTT RPM SNYQT UKHRP AASML AAYXX CITATION PHGZM PHGZT NPM 7XB 8FK AARCD AZQEC DWQXO GNUQQ K9. PJZUB PKEHL PPXIY PQEST PQGLB PQUKI PRINS 7X8 5PM PUEGO |
| ID | FETCH-LOGICAL-c492t-186e8c87ddd942105fdcf473cd22b0739416421a2527a927b542627f2f0fe5773 |
| IEDL.DBID | 7X7 |
| ISSN | 2052-4463 |
| IngestDate | Wed Aug 27 01:07:33 EDT 2025 Thu Aug 21 18:38:00 EDT 2025 Fri Jul 11 03:10:29 EDT 2025 Wed Aug 13 04:20:35 EDT 2025 Thu Apr 03 07:06:18 EDT 2025 Tue Jul 01 00:39:01 EDT 2025 Fri Feb 21 02:39:03 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | 2023. The Author(s). Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c492t-186e8c87ddd942105fdcf473cd22b0739416421a2527a927b542627f2f0fe5773 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Article-2 ObjectType-Undefined-1 ObjectType-Feature-3 content type line 23 |
| ORCID | 0000-0001-8397-334X 0000-0003-4743-9136 0000-0001-6314-3891 0000-0001-6617-9136 |
| OpenAccessLink | https://www.proquest.com/docview/2901290188?pq-origsite=%requestingapplication% |
| PMID | 38092796 |
| PQID | 2901290188 |
| PQPubID | 2041912 |
| PageCount | 17 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_edd6a19b27234b20a9751230252188b5 pubmedcentral_primary_oai_pubmedcentral_nih_gov_10719268 proquest_miscellaneous_2902954721 proquest_journals_2901290188 pubmed_primary_38092796 crossref_primary_10_1038_s41597_023_02793_w springer_journals_10_1038_s41597_023_02793_w |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2023-12-13 |
| PublicationDateYYYYMMDD | 2023-12-13 |
| PublicationDate_xml | – month: 12 year: 2023 text: 2023-12-13 day: 13 |
| PublicationDecade | 2020 |
| PublicationPlace | London |
| PublicationPlace_xml | – name: London – name: England |
| PublicationTitle | Scientific data |
| PublicationTitleAbbrev | Sci Data |
| PublicationTitleAlternate | Sci Data |
| PublicationYear | 2023 |
| Publisher | Nature Publishing Group UK Nature Publishing Group Nature Portfolio |
| Publisher_xml | – name: Nature Publishing Group UK – name: Nature Publishing Group – name: Nature Portfolio |
| References | Futrell (CR30) 2021; 55 CR14 Ferreira, Silva, Curado, Teixeira (CR8) 2022; 22 CR13 CR12 CR11 Livingstone, Russo (CR21) 2018; 13 Gonzalez-Lopez, Gomez-Alanis, Martín Doñas, Pérez-Córdoba, Gomez (CR3) 2020; 8 Cooke, Barker, Cunningham, Shao (CR27) 2006; 120 Titze (CR16) 1989; 85 Ge (CR17) 2023 Abdrakhmanova (CR22) 2021; 21 Wang, Zhang, Wu, Kong, Zhang (CR25) 2021; 11 Cester (CR10) 2022; 13 CR2 Hao, Zhu, Wang, Yang, Zeng (CR20) 2023; 23 Ozturk, Wu, Wang, Liu (CR6) 2023; 10 Bednar, Lalor (CR4) 2020; 205 CR5 Wang, Muñoz-Ferreras, Gu, Li, Gómez-García (CR19) 2014; 62 Busso (CR28) 2008; 42 CR29 CR9 CR26 Cai, Zheng, Luo (CR1) 2022; 24 CR23 Hameed (CR7) 2022; 13 Li, Shrestha, Heidari, Le Kernec, Fioranelli (CR15) 2020; 20 Kim, Lee (CR18) 2019; 19 Ozturk, Wu, Wang, Wu, Liu (CR24) 2023; 31 JA Gonzalez-Lopez (2793_CR3) 2020; 8 M Abdrakhmanova (2793_CR22) 2021; 21 A Bednar (2793_CR4) 2020; 205 S Kim (2793_CR18) 2019; 19 D Ferreira (2793_CR8) 2022; 22 Y Wang (2793_CR25) 2021; 11 2793_CR11 L Cester (2793_CR10) 2022; 13 Y Ge (2793_CR17) 2023 H Hameed (2793_CR7) 2022; 13 2793_CR14 2793_CR13 2793_CR12 G Wang (2793_CR19) 2014; 62 C Cai (2793_CR1) 2022; 24 C Busso (2793_CR28) 2008; 42 X Hao (2793_CR20) 2023; 23 M Cooke (2793_CR27) 2006; 120 MZ Ozturk (2793_CR24) 2023; 31 MZ Ozturk (2793_CR6) 2023; 10 S Livingstone (2793_CR21) 2018; 13 2793_CR26 IR Titze (2793_CR16) 1989; 85 2793_CR2 2793_CR23 2793_CR29 R Futrell (2793_CR30) 2021; 55 2793_CR5 H Li (2793_CR15) 2020; 20 2793_CR9 |
| References_xml | – volume: 10 start-page: 4431 year: 2023 end-page: 4448 ident: CR6 article-title: RadioMic: Sound sensing via radio signals publication-title: IEEE Internet of Things Journal doi: 10.1109/JIOT.2022.3217968 – volume: 13 start-page: 1 year: 2022 end-page: 9 ident: CR7 article-title: Pushing the limits of remote RF sensing by reading lips under the face mask publication-title: Nature communications doi: 10.1038/s41467-022-32231-1 – ident: CR14 – volume: 55 start-page: 63 year: 2021 end-page: 77 ident: CR30 article-title: The natural stories corpus: a reading-time corpus of english texts containing rare syntactic constructions publication-title: Language Resources and Evaluation doi: 10.1007/s10579-020-09503-7 – ident: CR2 – ident: CR12 – volume: 120 start-page: 2421 year: 2006 end-page: 2424 ident: CR27 article-title: An audio-visual corpus for speech perception and automatic speech recognition publication-title: The Journal of the Acoustical Society of America doi: 10.1121/1.2229005 – volume: 11 start-page: 9870 year: 2021 ident: CR25 article-title: Speckle noise detection and removal for laser speech measurement systems publication-title: Applied Sciences doi: 10.3390/app11219870 – ident: CR29 – volume: 31 start-page: 1333 year: 2023 end-page: 1347 ident: CR24 article-title: Radioses: mmWave-based audioradio speech enhancement and separation system publication-title: IEEE/ACM Transactions on Audio, Speech, and Language Processing doi: 10.1109/TASLP.2023.3250846 – volume: 23 start-page: 330 year: 2023 ident: CR20 article-title: A speech enhancement algorithm for speech reconstruction based on laser speckle images publication-title: Sensors doi: 10.3390/s23010330 – ident: CR23 – volume: 13 start-page: 3743 year: 2022 end-page: 3750 ident: CR10 article-title: Remote laser-speckle sensing of heart sounds for health assessment and biometric identification publication-title: Biomedical Optics Express doi: 10.1364/BOE.451416 – volume: 42 start-page: 335 year: 2008 end-page: 359 ident: CR28 article-title: Iemocap: Interactive emotional dyadic motion capture database publication-title: Language resources and evaluation doi: 10.1007/s10579-008-9076-6 – volume: 205 start-page: 116283 year: 2020 ident: CR4 article-title: Where is the cocktail party? Decoding locations of attended and unattended moving sound sources using EEG publication-title: NeuroImage doi: 10.1016/j.neuroimage.2019.116283 – volume: 8 start-page: 177995 year: 2020 end-page: 178021 ident: CR3 article-title: Silent speech interfaces for speech restoration: A Review publication-title: IEEE Access doi: 10.1109/ACCESS.2020.3026579 – year: 2023 ident: CR17 publication-title: figshare doi: 10.6084/m9.figshare.c.6942846.v1 – volume: 22 start-page: 649 year: 2022 ident: CR8 article-title: Exploring silent speech interfaces based on frequency-modulated continuous-wave radar publication-title: Sensors doi: 10.3390/s22020649 – volume: 20 start-page: 1191 year: 2020 end-page: 1201 ident: CR15 article-title: Bi-LSTM network for multimodal continuous human activity recognition and fall detection publication-title: IEEE Sensors Journal doi: 10.1109/JSEN.2019.2946095 – volume: 62 start-page: 1387 year: 2014 end-page: 1399 ident: CR19 article-title: Application of linear-frequency-modulated continuous-wave (lfmcw) radars for tracking of vital signs publication-title: IEEE Transactions on Microwave Theory and Techniques doi: 10.1109/TMTT.2014.2320464 – volume: 19 start-page: 2205 year: 2019 end-page: 2216 ident: CR18 article-title: Low-Complexity joint extrapolation-MUSIC-based 2-D parameter estimator for vital FMCW radar publication-title: IEEE Sensors Journal doi: 10.1109/JSEN.2018.2877043 – ident: CR13 – volume: 24 start-page: 432 year: 2022 end-page: 454 ident: CR1 article-title: Ubiquitous acoustic sensing on commodity IoT devices: A Survey publication-title: IEEE Communications Surveys Tutorials doi: 10.1109/COMST.2022.3145856 – ident: CR11 – volume: 21 start-page: 3465 year: 2021 ident: CR22 article-title: Speakingfaces: A large-scale multimodal dataset of voice commands with visual and thermal video streams publication-title: Sensors doi: 10.3390/s21103465 – ident: CR9 – ident: CR5 – volume: 85 start-page: 1699 year: 1989 end-page: 1707 ident: CR16 article-title: Physiologic and acoustic differences between male and female voices publication-title: The Journal of the Acoustical Society of America doi: 10.1121/1.397959 – ident: CR26 – volume: 13 start-page: e0196391 year: 2018 ident: CR21 article-title: Ryerson audiovisual database of emotional speeches and songs (ravdess): a dynamic, multimodal set of north american english face and voice expressions publication-title: Plos One doi: 10.1371/journal.pone.0196391 – ident: 2793_CR12 doi: 10.1016/j.procs.2021.12.222 – volume: 13 start-page: 1 year: 2022 ident: 2793_CR7 publication-title: Nature communications doi: 10.1038/s41467-022-32231-1 – volume: 55 start-page: 63 year: 2021 ident: 2793_CR30 publication-title: Language Resources and Evaluation doi: 10.1007/s10579-020-09503-7 – ident: 2793_CR13 doi: 10.1145/3277883.3277884 – volume: 21 start-page: 3465 year: 2021 ident: 2793_CR22 publication-title: Sensors doi: 10.3390/s21103465 – volume: 19 start-page: 2205 year: 2019 ident: 2793_CR18 publication-title: IEEE Sensors Journal doi: 10.1109/JSEN.2018.2877043 – ident: 2793_CR9 doi: 10.1145/3539490.3539599 – volume: 8 start-page: 177995 year: 2020 ident: 2793_CR3 publication-title: IEEE Access doi: 10.1109/ACCESS.2020.3026579 – volume: 85 start-page: 1699 year: 1989 ident: 2793_CR16 publication-title: The Journal of the Acoustical Society of America doi: 10.1121/1.397959 – volume: 42 start-page: 335 year: 2008 ident: 2793_CR28 publication-title: Language resources and evaluation doi: 10.1007/s10579-008-9076-6 – ident: 2793_CR2 doi: 10.1016/j.specom.2007.02.006 – ident: 2793_CR26 doi: 10.1109/TNSRE.2023.3266299 – volume: 13 start-page: 3743 year: 2022 ident: 2793_CR10 publication-title: Biomedical Optics Express doi: 10.1364/BOE.451416 – volume: 20 start-page: 1191 year: 2020 ident: 2793_CR15 publication-title: IEEE Sensors Journal doi: 10.1109/JSEN.2019.2946095 – ident: 2793_CR23 doi: 10.1145/3447993.3448626 – volume: 11 start-page: 9870 year: 2021 ident: 2793_CR25 publication-title: Applied Sciences doi: 10.3390/app11219870 – ident: 2793_CR29 doi: 10.1109/SLT48900.2021.9383619 – volume: 22 start-page: 649 year: 2022 ident: 2793_CR8 publication-title: Sensors doi: 10.3390/s22020649 – ident: 2793_CR14 doi: 10.1145/3397321 – volume: 24 start-page: 432 year: 2022 ident: 2793_CR1 publication-title: IEEE Communications Surveys Tutorials doi: 10.1109/COMST.2022.3145856 – volume: 205 start-page: 116283 year: 2020 ident: 2793_CR4 publication-title: NeuroImage doi: 10.1016/j.neuroimage.2019.116283 – volume: 31 start-page: 1333 year: 2023 ident: 2793_CR24 publication-title: IEEE/ACM Transactions on Audio, Speech, and Language Processing doi: 10.1109/TASLP.2023.3250846 – ident: 2793_CR5 doi: 10.1145/3485730.3485945 – year: 2023 ident: 2793_CR17 doi: 10.6084/m9.figshare.c.6942846.v1 – volume: 120 start-page: 2421 year: 2006 ident: 2793_CR27 publication-title: The Journal of the Acoustical Society of America doi: 10.1121/1.2229005 – volume: 23 start-page: 330 year: 2023 ident: 2793_CR20 publication-title: Sensors doi: 10.3390/s23010330 – ident: 2793_CR11 doi: 10.1109/CVPR46437.2021.00500 – volume: 62 start-page: 1387 year: 2014 ident: 2793_CR19 publication-title: IEEE Transactions on Microwave Theory and Techniques doi: 10.1109/TMTT.2014.2320464 – volume: 10 start-page: 4431 year: 2023 ident: 2793_CR6 publication-title: IEEE Internet of Things Journal doi: 10.1109/JIOT.2022.3217968 – volume: 13 start-page: e0196391 year: 2018 ident: 2793_CR21 publication-title: Plos One doi: 10.1371/journal.pone.0196391 |
| SSID | ssj0001340570 |
| Score | 2.2920346 |
| Snippet | Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech recognition. Our... Abstract Small-scale motion detection using non-invasive remote sensing techniques has recently garnered significant interest in the field of speech... |
| SourceID | doaj pubmedcentral proquest pubmed crossref springer |
| SourceType | Open Website Open Access Repository Aggregation Database Index Database Publisher |
| StartPage | 895 |
| SubjectTerms | 639/166/985 639/166/987 639/766/930/1032 Data Descriptor Datasets Humanities and Social Sciences Lip Lipreading Motion detection multidisciplinary Remote sensing Science Science (multidisciplinary) Speech Speech recognition Voice recognition |
| SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NT9wwEB1VnLhUfLWEL7lSD60gIrGT2D5CVYSQ2hNI3Cw7dgQShNVmK_5-Z-zswlIQlx5yyJdivbHznjOTZ4CvoWkpf4PKrZBxCTOdW1faXOma-F5yGyij--t3c35VXVzX18-W-qKasGQPnIA7Dt43ttQObxOV44XVEjlKIFUjOSkX3UuR855NpuLXFUFCpBj_kimEOh6Qqch4lMe0pRb54xITRcP-11Tmv8WSLzKmkYjO1uDjqCDZSWr5OnwI_Qasj2N0YN9GI-nvm3B5wqhifBpuUpU6i9WD9w8eb6fK0CHMGGpWRuXqtsVOMwzs7nbCpqmwntneM3xhxvW-cCe5l2zB1dnPyx_n-biKQt5Wms_yUjVBtUp673WFE7y6821XSdF6zh3l6VCS4XGLgEqruXQ1mdTLjndFF2opxSdY6R_6sA0MX4xOIeUpJysktVp3KHCcrUPj6X_bOoPDOaJmkswyTExyC2US_gbxNxF_85jBKYG-uJKMruMBDL8Zw2_eC38Ge_OQmXH0DYZyw7QplcGXxWkcN5QMsX1A4OgaSnHiBDiDzynCi5YIVSAQuslALcV-qanLZ_rbm-jNjbNp1MwNPvho3k2e2vU2Fjv_A4tdWOXUv0uel2IPVmbTP2EfJdPMHcTR8RfJyQ3q priority: 102 providerName: Directory of Open Access Journals – databaseName: Scholars Portal Journals: Open Access dbid: M48 link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1Nb9QwEB2V9sIFUcpHoCBX6oEKAomdxPYBoYJaVZXaU1fqzbJjh1Yq2ZIsKvx7Zpxk0cLCIYd8Kc7zOPOcN54B2A9VTfoNMrdMxhJmOrUut6nSJfl7yW0gRffsvDqZFaeX5eUGTOWORgD7tVM7qic1627e_fj28yMO-A_DknH1vkcnRDlFeVQktUjv7sEWeiZJFQ3ORrof_7kIoifZuHZm_a0r_imm8V_HPf8OofxDR43u6fghPBh5JTscDGEbNkL7CLbHkduz12N66YMduDhkFEfehashdp3FmMKvc4-3U7xoHxYMmSyjIHZboyn1Pbu5vmXdEG7PbOsZfkZjFTDcGXKaPIbZ8dHF55N0rK2Q1oXmizRXVVC1kt57XeC0r2x83RRS1J5zR-odEjU8bjn2l9VcupJS18uGN1kTSinFE9hs5214Bgw_l06hI1ROFujqSt0g7XG2DJWnVbhlAm8mRM3tkELDROlbKDPgbxB_E_E3dwl8ItCXV1L663hg3n0x42gywfvK5tqhLYnC8cxqicRFIH9DxqIcPnJ36jIzmZQhxZg2pRLYW57G0UQSiW0DAkfXkPCJ0-IEng49vGyJUBkCoasE1ErfrzR19Ux7fRUzduMcG5l0hQ9-O5nJ73b9G4vn_3-NF3Cfk-XmPM3FLmwuuu_hJVKkhXsV7f4X0q4KYw priority: 102 providerName: Scholars Portal – databaseName: Springer Nature HAS Fully OA dbid: AAJSJ link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT9wwEB3xceGCChSa8iEjcWhVIrJ2EtvHbVWEVmovgMTNsmMHkNos2izi7zPjJIsW6KGHHBI7ijVje54zz88AJ6GsKH-DyC2T8QgznVo3sqnSBcV7yW2gjO6v3-XFdT65KW5WgA97YSJpP0paxml6YIedtRhoSDeUx6yjFunTKqyTVDv27fXxeHI5efmzIgiEZP0OmUyod15eikJRrP89hPmWKPkqWxqD0PkH2OzRIxt37d2CldBsw1Y_Plv2pReR_roDV2NGbPFZuOsY6iwyB_9OPb5OrNA2zBniVUZUdVthh2lb9uf-gc06Uj2zjWc4WcazvvCmUy75CNfnP69-XKT9CQpplWs-T0eqDKpS0nuvc1zcFbWv6lyKynPuKEeHcAyfW45esZpLV5BAvax5ndWhkFLswlozbcInYDgpOoXhTjmZY0ArdI3gxtkilJ722hYJfBssah46oQwTE9xCmc7-Bu1vov3NUwLfyeiLmiRyHR9MZ7emd7oJ3pd2pB32GJE7nlktEZ4IRGmIS5TDTx4MLjP9yGsN5YXpUiqB40UxjhlKhNgmoOGoDqU3cfGbwF7n4UVLhMrQELpMQC35fqmpyyXN_V3U5caVNOLlEj98OnSTl3b92xaf_6_6PmzQkfdEqRmJA1ibzx7DIQKjuTvqR8Izc9gGpQ priority: 102 providerName: Springer Nature |
| Title | A comprehensive multimodal dataset for contactless lip reading and acoustic analysis |
| URI | https://link.springer.com/article/10.1038/s41597-023-02793-w https://www.ncbi.nlm.nih.gov/pubmed/38092796 https://www.proquest.com/docview/2901290188 https://www.proquest.com/docview/2902954721 https://pubmed.ncbi.nlm.nih.gov/PMC10719268 https://doaj.org/article/edd6a19b27234b20a9751230252188b5 |
| Volume | 10 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3fb9MwED6N7YUXxBg_AqMyEg8giJbGSWw_oa7aNFXahGCT-mbZsbNNgrQ0Rfv3uXPcVoWNh6aqkyrO3dn32d_lDuC9r2ribxC5ZSKUMFOpsUOTSlWSvxe58cTonl9UZ1fFZFpO44ZbF8MqV3NimKjdrKY98iPi--gj5Zf5r5SqRhG7GktoPII9Sl1GVi2mYrPHwgmOZPFdmYzLow79FaUfzQN5qXh6t-WPQtr--7DmvyGTf_GmwR2dPoUnEUeyUa_4fdjx7TPYjyO1Yx9iOumPB3A5YhQ3vvA3faw6CzGEP2cO_07xoZ1fMkSujILWTY2m03Xsx-2cLfrwemZax3DaDFW_8Eefw-Q5XJ2eXI7P0lhLIa0LlS_Toay8rKVwzqkCl3ll4-qmELx2eW6JrUNghu0mR_0YlQtbUqp60eRN1vhSCP4CdttZ618Bw-nRSnR80ooCXVupGoQ51pS-cvTWbZnAp5VE9bxPmaED1c2l7uWvUf46yF_fJXBMQl9fSemuQ8Nsca3j6NHeucoMlUXb4YXNM6MEAhWOeA0RirR4y8OVynQcg53eWEwC79ancfQQJWJaj4Kja4joxGVwAi97Da97wmWGglBVAnJL91td3T7T3t6EDN24pkbkXOGNP6_MZNOvh2Xx-v-P8QYeU7F7CqYZ8kPYXS5--7cIiZZ2EOx-AHuj0eT7BL-PTy6-fsPWcTUehG0GPJ4X8g8hbwzC |
| linkProvider | ProQuest |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9NAEB6VcoALorzqUmCRQAKBVWf92N0DQuVRpfRxSqXcll3vmlYCJ42DIv4Uv5GZtZ0ovG495BDbidez8_jW3-wMwDNflMTfIHJLRGhhpmJjByaWKqd4L7jxxOienBbDs-zTOB9vwM9-LwylVfY-MThqNynpHfke8X30kfLt9DKmrlHErvYtNFq1OPI_Frhka94cfsD5fc75wcfR-2HcdRWIy0zxeTyQhZelFM45leGCJ69cWWUiLR3nlngrhCh43HAcqVFc2JyKtouKV0nlcyFS_N9rcB0Db0KLPTEWq3c6KcGfpNubk6Ryr8H4SOVOeSBLVRov1uJfaBPwN2z7Z4rmbzxtCH8Ht-FWh1vZfqtoW7Dh6zuw1XmGhr3oyle_vAujfUZ56jN_3ubGs5Cz-G3i8OeUj9r4OUOkzChJ3pSoqk3Dvl5M2axN52emdgzddOgyhl_amin34OxKpHwfNutJ7beBoTu2EgOttCLDUJqrCmGVNbkvHO3yzSN41UtUT9sSHTpQ66nUrfw1yl8H-etFBO9I6Msrqbx2ODCZfdGdtWrvXGEGyqKuppnliVECgVGK-BARkbR4y91-ynRn841eaWgET5en0VqJgjG1R8HRNUSs4rI7ggftDC9HksoEBaGKCOTa3K8Ndf1MfXEeKoKjKiJSL_DGr3s1WY3r37LY-f9jPIEbw9HJsT4-PD16CDc5afGAx4N0Fzbns-_-EcKxuX0cbIDB56s2ul-SDUHH |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3db9MwED-NTkK8IMZnxgAjgQSCaInzYecBoY2t2hhUE9qkvRk7dtikkZamqOJf46_jzklala-3PfShSdo45_Pd7_I73wE8c3lJ_A0it0j4FmZFqE2sQ1lk5O8F144Y3Y-j_OA0fX-Wna3Bz34vDKVV9jbRG2o7Lukd-TbxffTBgK3q0iKO94ZvJ99C6iBFTGvfTqNVkSP3Y47hW_PmcA_n-jnnw_2Tdwdh12EgLNOCz8JY5k6WUlhrixSDn6yyZZWKpLScG-KwEK7gcc1x1LrgwmRUwF1UvIoqlwmR4P9eg3VBUdEA1nf3R8eflm94EgJDUbdTJ0rkdoPekoqfck-dFkk4X_GGvmnA35Dunwmbv7G23hkOb8HNDsWynVbtNmDN1bdho7MTDXvRFbN-eQdOdhhlrU_deZspz3wG49exxZ9TdmrjZgxxM6OUeV2i4jYNu7yYsGmb3M90bRkabd9zDL-0FVTuwumVyPkeDOpx7R4AQ-NsJLpdaUSKjjUrKgRZRmcut7TnNwvgVS9RNWkLdihPtCdStfJXKH_l5a_mAeyS0BdXUrFtf2A8_aK6tauctbmOC4Oam6SGR7oQCJMSRIuIj6TBW271U6Y6C9Copb4G8HRxGtcuETK6dig4uoZoVgzCA7jfzvBiJImMUBBFHoBcmfuVoa6eqS_OfX1wjOgRt-d449e9mizH9W9ZbP7_MZ7AdVxw6sPh6Ogh3OCkxDEP42QLBrPpd_cIsdnMPO4WAYPPV73ufgFQO0di |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+comprehensive+multimodal+dataset+for+contactless+lip+reading+and+acoustic+analysis&rft.jtitle=Scientific+data&rft.au=Ge%2C+Yao&rft.au=Tang%2C+Chong&rft.au=Li%2C+Haobo&rft.au=Chen%2C+Zikang&rft.date=2023-12-13&rft.pub=Nature+Publishing+Group&rft.eissn=2052-4463&rft.volume=10&rft.issue=1&rft.spage=895&rft_id=info:doi/10.1038%2Fs41597-023-02793-w&rft.externalDBID=HAS_PDF_LINK |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2052-4463&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2052-4463&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2052-4463&client=summon |