Deep Neural Network for User Attribute Recognition in Metaverse Hand Images

This work develops a Deep Neural Network (DNN) that can recognize user attributes such as gender, age group, skin color, accessories, nail polish, palm and back of the hand, and left and right hands from hand images to understand user attributes of virtual-real interactions in metaverse. The propose...

Full description

Saved in:
Bibliographic Details
Published in2023 International Conference on Computer, Information and Telecommunication Systems (CITS) pp. 01 - 06
Main Authors Chen, Oscal T.-C., Yan, Li-Cheng
Format Conference Proceeding
LanguageEnglish
Published IEEE 10.07.2023
Subjects
age-group recognition
Artificial neural networks
attention mechanism
deep neural network
Feature extraction
gender identification
image classification
Image color analysis
Image recognition
Metaverse
multi- attribute identification
Nails
Training data
Online AccessGet full text

Cover

More Information
Summary:This work develops a Deep Neural Network (DNN) that can recognize user attributes such as gender, age group, skin color, accessories, nail polish, palm and back of the hand, and left and right hands from hand images to understand user attributes of virtual-real interactions in metaverse. The proposed DNN is based on DenseNet-169 in which outputs from four dense blocks are convoluted and concatenated to form multi-scale features. These features then go through the spatial coordinate attention module, which emphasizes meaningful parts. To deal with unbalanced data, loss scaling parameters and dynamically learnable weighted loss function are adopted along with data augmentation to improve the performance. In particular, dorsal hand images are better suited for classifying gender, age group, and skin color than palmar images due to the presence of abundant skin textures, wrinkles, age spots, veins and so on. Accordingly, the proposed DNN includes a post- processing module to classify these three attributes in terms of dorsal hands. Our DNN achieves superior performance in gender identification with 89.74% accuracy using NTU-PI v1, 94.03% and 92.69% in dorsal and palmar hand recognition using 11K hands, respectively, and an average multi-attribute accuracy of 93.17% on 11K hands. Therefore, the DNN proposed herein can be widely applied to various metaverse applications to provide users with customized service experiences.
DOI:10.1109/CITS58301.2023.10188725