A Chinese Calligraphy-Writing Robotic System Based on Image-to-Action Translations and a Hypothesis Generation Net

• Published in: IEEE access Vol. 11; p. 1
• Main Authors: Hsu, Min-Jie, Yeh, Po-Chao, Chien, Yi-Hsing, Lu, Cheng-Kai, Wang, Wei-Yen, Hsu, Chen-Chien
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Calligraphy; China; Chinese calligraphy; Error correction; Hypotheses; hypothesis generation net; Image processing; image-to-action translation; Learning; Machine translation; Manipulators; Mathematical analysis; Matrix converters; Modules; Robot kinematics; robotic calligraphy system; Robots; Trajectories; Translations; Writing
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2023.3252902

This paper attempts to use a delta robot's structure and reliable coordinates to develop a self-learning Chinese calligraphy-writing system that requires precise control. Ideally, to achieve human-like behavior, a delta robot can learn stroke trajectories autonomously and present the stroke beauty of calligraphy characters. Unfortunately, state-of-the-art approaches have not yet considered the presentation of stroke beauty resulting from angles of rotation and tilt of the brush. This paper presents an integrated system consisting of a stroke processing module, a hypothesis generation net (HGN) learning model, a delta robot, and an image capture module. Our approach utilizes both the stroke trajectories from the stroke processing module and angles information from the HGN learning model to automatically produce five degrees of freedom action instructions. Based on the instructions, the delta robot completes calligraphy writing. Then, the image capture module provides feedback to the writing system for error calculation and coordinate correction. We utilize the mean absolute percentage error to verify the performance of the writing results. A correction algorithm and linear regression were used to improve the error correction results (less than 2% error). After several cycles, the written results approached the target sample finally. Consequently, the written results produced by the delta robot prove that our proposed system is capable of self-learning and correction.