Cross-Modal Information Recovery and Enhancement Using Multiple-Input-Multiple-Output Variational Autoencoder

• Published in: IEEE internet of things journal Vol. 11; no. 15; pp. 26470 - 26480
• Main Author: Liang, Jessica E.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.08.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Audio data; Autoencoders; Brain; Coders; Complexity; Cross-modal information processing; Data integration; Data processing; Decoders; Decoding; Image reconstruction; Information theory; Internet of Things; Linearization; Machine learning; MIMO; multiple-input–multiple-output (MIMO); Recovery; Robot sensing systems; Robots; variational autoencoder (VAE); Vertebrates; Visualization

Motivated by the cross-modal information processing mechanism of human brain, vertebrates, and invertebrates, we propose a multiple-input-multiple-output (MIMO) variational autoencoder (VAE) and subsequently apply it to cross-modal information recovery and enhancement. For a cross-modal system with...