Dataset distillation as an enabling technique for on-device training in TinyML for IoT: an RFID use case

Enabling decision making at the far edge involves the capillarity distribution of Machine Learning models on resource-constrained devices with limited memory and computational capabilities. In this context, implementing the training phase on the device itself is challenging for the aforementioned li...

Full description

Saved in:
Bibliographic Details
Published in2023 8th International Conference on Smart and Sustainable Technologies (SpliTech) pp. 1 - 4
Main Authors Accettola, Andrea Giovanni, Merenda, Massimo
Format Conference Proceeding
LanguageEnglish
Published University of Split, FESB 20.06.2023
Subjects
Data integrity
Data models
dataset distillation
Decision making
Dimensionality reduction
few-shot learning
Histograms
knowledge distillation
on-device training
Performance evaluation
resource-constrained devices
RFID
tinyML
Training
Online AccessGet full text

Cover

More Information
Summary:Enabling decision making at the far edge involves the capillarity distribution of Machine Learning models on resource-constrained devices with limited memory and computational capabilities. In this context, implementing the training phase on the device itself is challenging for the aforementioned limitations. The utilization of knowledge distillation as enabling technique for single- or few-shot on-device training on resource-constrained devices is presented in this work. With the combination of histogram analysis and subsequent dimensionality reduction techniques, we show in this work an effective technique of dataset distillation that leads to condensate the information in very few observations. The use of these techniques on a dataset built of measurements from Radio Frequency Identification (RFID) devices is the subject of a case study that we provide. Our findings demonstrate that the suggested approach performance is comparable with state-of-the-art models while utilizing a sizable reduction in processing power and memory. The contributions of this work are twofold. Firstly, we provide a novel approach for on-device training which is particularly relevant for resource-constrained devices. Secondly, we present an application of the proposed method to a challenging dataset of RFID data, demonstrating its effectiveness in a real-world scenario.
DOI:10.23919/SpliTech58164.2023.10193138