A developmental model of audio-visual attention (MAVA) for bimodal language learning in infants and robots

A social individual needs to effectively manage the amount of complex information in his or her environment relative to his or her own purpose to obtain relevant information. This paper presents a neural architecture aiming to reproduce attention mechanisms (alerting/orienting/selecting) that are ef...

Full description

Saved in:
Bibliographic Details
Published inScientific reports Vol. 14; no. 1; pp. 20492 - 9
Main Authors Bergoin, Raphaël, Boucenna, Sofiane, D’Urso, Raphaël, Cohen, David, Pitti, Alexandre
Format Journal Article
LanguageEnglish
Published London Nature Publishing Group UK 03.09.2024
Nature Publishing Group
Nature Portfolio
Subjects
631/378
631/477
639/166
Artificial Intelligence
Attention
Attention - physiology
Auditory Perception - physiology
Cognitive science
Computer Science
Humanities and Social Sciences
Humans
Infant
Information systems
Language
Language Development
Learning - physiology
Mouth
multidisciplinary
Neural networks
Newborn babies
Robotics
Robotics - methods
Robots
Science
Science (multidisciplinary)
Sensory integration
Social discrimination learning
Sound
Visual discrimination learning
Visual perception
Visual Perception - physiology
Visual stimuli
Online AccessGet full text

Cover

More Information
Summary:A social individual needs to effectively manage the amount of complex information in his or her environment relative to his or her own purpose to obtain relevant information. This paper presents a neural architecture aiming to reproduce attention mechanisms (alerting/orienting/selecting) that are efficient in humans during audiovisual tasks in robots. We evaluated the system based on its ability to identify relevant sources of information on faces of subjects emitting vowels. We propose a developmental model of audio-visual attention (MAVA) combining Hebbian learning and a competition between saliency maps based on visual movement and audio energy. MAVA effectively combines bottom-up and top-down information to orient the system toward pertinent areas. The system has several advantages, including online and autonomous learning abilities, low computation time and robustness to environmental noise. MAVA outperforms other artificial models for detecting speech sources under various noise conditions.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-024-69245-2