Synthetic Image Generation With a Fine-Tuned Latent Diffusion Model for Organ on Chip Cell Image Classification

• Published in: 2023 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA) pp. 148 - 153
• Main Authors: Ivanovs, Maksims, Leja, Laura, Zviedris, Karlis, Rimsa, Roberts, Narbute, Karina, Movcana, Valerija, Rumnieks, Felikss, Strods, Arnis, Gillois, Kevin, Mozolevskis, Gatis, Abols, Arturs, Kadikis, Roberts
• Format: Conference Proceeding
• Language: English
• Published: Division of Signal Processing and Electronic Systems, Poznan University of Technology (DSPES PUT) 20.09.2023
• Subjects: Adaptation models; Artificial neural networks; Biological system modeling; Image synthesis; Microscopy; Signal processing algorithms; Training
• ISSN: 2326-0319
• DOI: 10.23919/SPA59660.2023.10274460

Augmentation of the datasets of authentic microscopy images with synthetic images is a promising solution to the problem of the limited availability of biomedical data for training deep neural network (DNN) based classifiers. In the present study, we use a text-to-image latent stable diffusion model fine-tuned by means of low-rank adaptation (LoRA) to augment a small dataset of the images of organ on chip cells. While the resulting synthetic images appear quite similar to the authentic images on which the low-rank adaptation was performed, we find that neither training the EfficientNetB7 DNN model solely on the synthetic data nor augmentation of the real-world dataset with different proportions (10, 25, 50, and 75 percent) of these data leads to the improvement of the accuracy of the model. The findings of our study suggest that a further exploration of the low-rank adaptation options is needed to fully use the capacity of latent diffusion models for the synthesis of biomedical images.