Human body behavior detection method and system based on two-dimensional and three-dimensional CNN (Convolutional Neural Network)

• Main Authors: ZHU HONGTAO, ZHANG ZHENG, XIE SHATING, WEI YUN, LIU JIE, DOU FEI, BAI WENFEI, ZHAO LIYUAN, TIAN QING, ZANG SHUO, NING YAO
• Format: Patent
• Language: Chinese; English
• Published: 27.09.2022
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

The invention provides a human body behavior detection method and system based on a two-dimensional CNN and a three-dimensional CNN. An input module receives a video frame sequence and extracts key frames in the video frame sequence, a data enhancement module carries out sample size expansion on input frame images, and a two-dimensional convolution model and a three-dimensional convolution model are used for respectively extracting spatial features contained in the key frames and spatial-temporal features contained in the video frame sequence. And after feature fusion is carried out on the two features, a final feature map D obtained by fusion passes through a 1 * 1 convolution layer to generate a required channel number, and optimization processing is completed through a loss function to obtain a detection model. According to the method, the human body behavior detection model is constructed through the Gram matrix, the dot product of each flattened feature vector in the whole feature map can be considered t