COVID-19 detection from thermal image and tabular medical data utilizing multi-modal machine learning

• Published in: 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS) pp. 646 - 653
• Main Authors: Ul Alam, Mahbub, Hollmen, Jaakko, Rahmani, Rahim
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2023
• Subjects: Boosting; Convolutional neural networks; COVID-19; COVID-19 Detection; Deep learning; Internet of Medical Things; Machine Learning; Multi-Modality; Multilayer perceptrons; Real-time systems; Tabular Medical Data; Thermal factors; Thermal Image
• ISSN: 2372-9198
• DOI: 10.1109/CBMS58004.2023.00294

COVID-19 is a viral infectious disease that has created a global pandemic, resulting in millions of deaths and disrupting the world order. Different machine learning and deep learning approaches were considered to detect it utilizing different medical data. Thermal imaging is a promising option for detecting COVID-19 as it is low-cost, non-invasive, and can be maintained remotely. This work explores the COVID-19 detection issue using the thermal image and associated tabular medical data obtained from a publicly available dataset. We incorporate a multi-modal machine learning approach where we investigate the different combinations of medical and data type modalities to get an improved result. We use different machine learning and deep learning methods, namely random forests, Extreme Gradient Boosting (XGBoost), Multilayer Perceptron (MLP), and Convolutional Neural Network (CNN). Overall multi-modal results outperform any single modalities, and it is observed that the thermal image is a crucial factor in achieving it. X G Boost provided the best result with the area under the receiver operating characteristic curve (AUROC) score of 0.91 and the area under the precision-recall curve (AUPRC) score of 0.81. We also report the average of leave-one-positive-instance-out cross- validation evaluation scores. This average score is consistent with the test evaluation score for random forests and XGBoost methods. Our results suggest that utilizing thermal image with associated tabular medical data could be a viable option to detect COVID-19, and it should be explored further to create and test a real-time, secure, private, and remote COVID-19 detection application in the future.