Joint Embedding of Deep Visual and Semantic Features for Medical Image Report Generation

• Published in: IEEE transactions on multimedia Vol. 25; pp. 167 - 178
• Main Authors: Yang, Yan, Yu, Jun, Zhang, Jian, Han, Weidong, Jiang, Hanliang, Huang, Qingming
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Coders; Decoding; deep neural network; Diagnosis; Embedding; encoder-decoder framework; Encoders-Decoders; Encoding; Feature extraction; Medical diagnostic imaging; Medical image report generation; Medical imaging; Natural language (computers); Semantics; Sentences; Task analysis; visual-semantic joint embedding; Visualization

Medical image report generation (MeIRG) aims at generating associated diagnosis descriptions with natural language sentences from medical images, which is essential in the computer-aided diagnosis system. Nevertheless, this task remains challenging in that medical images and linguistic expressions s...