SOCIAL RELATION ANALYSIS METHOD AND SYSTEM BASED ON MULTI-MODAL DATA, AND STORAGE MEDIUM

• Main Authors: CHEN, Cheng, ZHAO, Jianqiang, PENG, Chuang, ZHANG, Hui, CHEN, Simeng, HAN, Mingxi
• Format: Patent
• Language: Chinese; English; French
• Published: 15.02.2024
• Subjects: CALCULATING; COMPUTING; COUNTING; DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FORADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORYOR FORECASTING PURPOSES; PHYSICS; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE,COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTINGPURPOSES, NOT OTHERWISE PROVIDED FOR

The present invention provides a social relation analysis method based on multi-modal data. The method comprises: S1, extracting social texts and social image information of personnel, respectively converting same into text features and image features, collecting statistics about personnel intimacy, and constructing a personnel social network graph on the basis of the personnel intimacy; S2, inputting the text features and the image features into a transformer-based multi-modal fusion model to obtain fused features; and S3, analyzing the personnel social network graph by using a Si-SCAN graph clustering algorithm to obtain a social relation clustering result, wherein the Si-SCAN graph clustering algorithm is constructed by introducing the personnel intimacy and fused feature information on the basis of the SCAN algorithm. According to the present invention, social relations are deeply analyzed on the basis of information of text and image modalities. By means of the design of the multi-modal information fusio