SYSTEMS, METHODS, AND APPARATUSES FOR IMPLEMENTING SELF-SUPERVISED VISUAL REPRESENTATION LEARNING USING ORDER AND APPEARANCE RECOVERY ON A VISION TRANSFORMER

• Main Authors: Ma, DongAo, Liang, Jiangming, Pang, Jiaxuan
• Format: Patent
• Language: English
• Published: 28.09.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATIONTECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING ORPROCESSING OF MEDICAL OR HEALTHCARE DATA; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTEDFOR SPECIFIC APPLICATION FIELDS; PHYSICS

Described herein are means for performing self-supervised visual representation learning using order and appearance recovery on a vision transformer. An exemplary system having a processor and memory is specially configured to execute instructions including: receiving medical image training data; selecting a medical image; generating a first perturbed image by applying local pixel shuffling and other image perturbations and outputting a first patchified perturbed image; generating a second randomized patchified image by patchifying and applying a random permutation to the original image; inputting the first patchified perturbed image and the second randomized patchified image into first and second transformer encoders which each generate and then share first and second generated weights through the recovery of both and patch order appearance from each image; and outputting a pre-trained AI model to perform medical image diagnosis on a new medical image absent from the training data input received by the system.