Visual behavior recognition method and system based on text semantic supervision and computer readable medium

• Main Authors: ZHANG LIANG, SONG JUAN, ZHANG WEIWEI, FENG MINGTAO, ZHU GUANGMING, SHEN PEIYI
• Format: Patent
• Language: Chinese; English
• Published: 30.03.2021
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention discloses a visual behavior recognition method and system based on text semantic supervision and a computer readable medium. The method comprises the steps of text semantic feature extraction, visual feature extraction based on text semantic supervision and visual behavior recognition construction. According to the method, text description normal forms of various behaviors are concluded on the basis of text description sentences of a video sample set of the same type of behaviors, a sample pair data set is constructed, and action semantic feature vectors and relation semantic feature vectors of the text description sentences are extracted from a text semantic extraction model; the extracted action visual feature vector and the relation visual feature vector are supervised byusing the action semantic feature vector and the relation semantic feature vector, and behavior recognition is performed by using the extracted action visual feature vector and the relation visual feature vector; therefore, t