Modal alignment model training method and device, modal alignment method and device and storage medium

• Main Author: ZHAI BINXU
• Format: Patent
• Language: Chinese; English
• Published: 07.11.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention relates to a modal alignment model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: inputting first training modal information and second training modal information into an initial modal alignment model for modal alignment characterization to obtain a training modal alignment characterization vector; calculating a modal similarity degree between a first modal alignment characterization vector in the training modal alignment characterization vectors and a second modal alignment characterization vector in the training modal alignment characterization vectors; calculating a probability distribution distance between the first modal alignment representation vector and the second modal alignment representation vector based on the modal similarity degree; the initial modal alignment model is reversely updated based on the probability distribution distance, loop iteration is carried out, a first target modal alignm