Condition-Adaptive Graph Convolution Learning for Skeleton-Based Gait Recognition

• Published in: IEEE transactions on image processing Vol. 32; pp. 4773 - 4784
• Main Authors: Huang, Xiaohu, Wang, Xinggang, Jin, Zhidianqiu, Yang, Bo, He, Botao, Feng, Bin, Liu, Wenyu
• Format: Journal Article
• Language: English
• Published: New York IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; adaptive feature learning; Adaptive filters; Artificial neural networks; Body parts; Convolution; Feature extraction; Gait recognition; graph convolution; Learning; Legged locomotion; Modules; Network topologies; Skeleton; Skeleton-based gait recognition; Topology
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2023.3305822

Graph convolutional networks have been widely applied in skeleton-based gait recognition. A key challenge in this task is to distinguish the individual walking styles of different subjects across various views. Existing state-of-the-art methods employ uniform convolutions to extract features from diverse sequences and ignore the effects of viewpoint changes. To overcome these limitations, we propose a condition-adaptive graph (CAG) convolution network that can dynamically adapt to the specific attributes of each skeleton sequence and the corresponding view angle. In contrast to using fixed weights for all joints and sequences, we introduce a joint-specific filter learning (JSFL) module in the CAG method, which produces sequence-adaptive filters at the joint level. The adaptive filters capture fine-grained patterns that are unique to each joint, enabling the extraction of diverse spatial-temporal information about body parts. Additionally, we design a view-adaptive topology learning (VATL) module that generates adaptive graph topologies. These graph topologies are used to correlate the joints adaptively according to the specific view conditions. Thus, CAG can simultaneously adjust to various walking styles and viewpoints. Experiments on the two most widely used datasets (i.e., CASIA-B and OU-MVLP) show that CAG surpasses all previous skeleton-based methods. Moreover, the recognition performance can be enhanced by simply combining CAG with appearance-based methods, demonstrating the ability of CAG to provide useful complementary information.