PainterAR: A Self‐Painting AR Interface for Mobile Devices

• Published in: Computer animation and virtual worlds Vol. 35; no. 6
• Main Authors: Ma, Yuan, Shi, Yinghan, Zhao, Lizhi, Lu, Xuequan, Duh, Been‐Lirn, Wang, Meili
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.11.2024
• Subjects: Art works; Artificial intelligence; Augmented reality; Displays; human‐computer interaction; Interactive control; neural painting; rendering techniques; Virtual reality
• DOI: 10.1002/cav.2296
• ISSN: 1546-4261

ABSTRACT Painting is a complex and creative process that involves the use of various drawing skills to create artworks. The concept of training artificial intelligence models to imitate this process is referred to as neural painting. To enable ordinary people to engage in the process of painting, we propose PainterAR, a novel interface that renders any paintings stroke‐by‐stroke in an immersive and realistic augmented reality (AR) environment. PainterAR is composed of two components: the neural painting model and the AR interface. Regarding the neural painting model, unlike previous models, we introduce the Kullback–Leibler divergence to replace the original Wasserstein distance existed in the baseline paint transformer model, which solves an important problem of encountering different scales of strokes (big or small) during painting. We then design an interactive AR interface, which allows users to upload an image and display the creation process of the neural painting model on the virtual drawing board. Experiments demonstrate that the paintings generated by our improved neural painting model are more realistic and vivid than previous neural painting models. The user study demonstrates that users prefer to control the painting process interactively in our AR environment. We propose PainterAR, a novel interface that renders any paintings stroke‐by‐stroke in an immersive and realistic Augmented Reality (AR) environment. PainterARis composed of two components: the neural painting model and the AR interface.