Method and device for generating picture with accurate characters according to character description and storage medium

• Main Author: SHI ZHEBIN
• Format: Patent
• Language: Chinese; English
• Published: 19.12.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

The invention relates to a method and device for generating a picture with accurate characters according to character description and a storage medium, and the method comprises the steps: a training stage: carrying out the BLIP and OCR of an image training sample, respectively extracting the text description of the image and the characters in the image, combining the two characters, and taking the combined characters as the character input of a potential diffusion model; in the inference stage, the use of the potential diffusion model is consistent with that of the text graph diffusion model, and a corresponding image can be generated by inputting text cues; image content understanding adopts BLIP, text description is generated for an image by using BILP, image text recognition adopts a paddle OCR technology to additionally extract text information in the image, a potential diffusion model LDMs is adopted as a base model for image generation, and a corresponding image is generated according to cue words; acco