Online Handwriting Recognition using LSTM on Microcontroller and IMU Sensors

The trend toward the Internet of Things has led to a rapid increase in the amount of data that needs to be processed. Artificial intelligence (AI) can serve as a very helpful tool to extract or compress essential information of data. However, AI places high demands on a system's hardware. This...

Full description

Saved in:
Bibliographic Details
Published in2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA) pp. 999 - 1004
Main Authors Meisl, Florian, Eibensteiner, Florian, Petz, Phillip, Langer, Josef
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.12.2022
Subjects
Edge Computing
Handwriting recognition
IMU Sensor
LSTM
Machine Learning
Microcontroller
Microcontrollers
Online Handwriting Recognition
Prototypes
Quantization (signal)
Real-time systems
Symbols
Training
Online AccessGet full text

Cover

More Information
Summary:The trend toward the Internet of Things has led to a rapid increase in the amount of data that needs to be processed. Artificial intelligence (AI) can serve as a very helpful tool to extract or compress essential information of data. However, AI places high demands on a system's hardware. This is not exactly in line with the strengths of embedded systems.This paper combines AI on embedded systems with the not-yet fully explored subject of online handwriting recognition (HWR). The main contribution is the deployment and real-time operation of AI on a microcontroller (MCU). Model architectures using long short-term memory (LSTM) cells and 1D convolutional neural networks (CNNs) are used to process live data from inertial measurement units (IMUs) sensors. The dataset used for training the AI models was recorded with a self-developed prototype. After training, the models are converted and deployed on a MCU. The conversion process includes quantization from a 32-bit floating-point to an 8-bit fixed-point datatype. The TensorFlow Lite Micro (TFLM) framework is used to run inference on the MCU. For predictions in real-time optimizations are applied to the framework, which results in running inference approx. 827 times faster. The optimized AI model implementation is then used to classify handwritten characters using the live data from the IMU sensors. This first approach has shown, that the separation of the symbols is necessary to be able to classify characters from live sensor data with high accuracy.
DOI:10.1109/ICMLA55696.2022.00167