HaSPeR: An Image Repository for Hand Shadow Puppet Recognition

• Published in: arXiv.org
• Main Authors: Syed Rifat Raiyan, Zibran Zarif Amio, Ahmed, Sabbir
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 19.08.2024
• Subjects: Applications programs; Computer vision; Datasets; Error analysis; Fingers; Flat surfaces; Illusions; Image classification; Mobile computing; Shadows; Statistical analysis
• ISSN: 2331-8422

Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling where hand shadows are projected onto flat surfaces to create illusions of living creatures. The skilled performers create these silhouettes by hand positioning, finger movements, and dexterous gestures to resemble shadows of animals and objects. Due to the lack of practitioners and a seismic shift in people's entertainment standards, this art form is on the verge of extinction. To facilitate its preservation and proliferate it to a wider audience, we introduce \({\rm H{\small A}SP{\small E}R}\), a novel dataset consisting of 8,340 images of hand shadow puppets across 11 classes extracted from both professional and amateur hand shadow puppeteer clips. We provide a detailed statistical analysis of the dataset and employ a range of pretrained image classification models to establish baselines. Our findings show a substantial performance superiority of traditional convolutional models over attention-based transformer architectures. We also find that lightweight models, such as MobileNetV2, suited for mobile applications and embedded devices, perform comparatively well. We surmise that such low-latency architectures can be useful in developing ombromanie teaching tools, and we create a prototype application to explore this surmission. Keeping the best-performing model InceptionV3 under the limelight, we conduct comprehensive feature-spatial, explainability, and error analyses to gain insights into its decision-making process. To the best of our knowledge, this is the first documented dataset and research endeavor to preserve this dying art for future generations, with computer vision approaches. Our code and data are publicly available.