Method and device for generating picture from text, electronic equipment and storage medium

• Main Authors: LIU FANGZHE, TONG XINZHE, CHANG LIYUAN
• Format: Patent
• Language: Chinese; English
• Published: 06.02.2024
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

The invention discloses a method and a device for generating a picture from a text, electronic equipment and a storage medium. The method comprises the following steps: acquiring a plurality of groups of training sample pairs; each group of training sample pairs comprises a text token and a picture token of one picture; applying the multiple groups of training sample pairs to train the autoregressive transformer network model, and determining a target picture output by the autoregressive transformer network model and an autoregressive transformer loss function; performing instance segmentation on the target picture by applying the instance segmentation network model to obtain an instance segmentation snapshot; calculating a content loss function based on the instance segmentation snapshot and the specified instance material; calculating a total loss function according to the autoregression transformer loss function and the content loss function; updating model parameters of the autoregression transformer netw