Reality mining and predictive analytics for building smart applications

• Published in: Journal of big data Vol. 6; no. 1; pp. 1 - 25
• Main Authors: Asri, Hiba, Mousannif, Hajar, Al Moatassime, Hassan
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 22.07.2019; Springer Nature B.V; SpringerOpen
• Subjects: Algorithms; Analytics; Applications programs; Big Data; Cell phones; Cellular telephones; Clustering; Communications Engineering; Computational Science and Engineering; Computer Science; Data management; Data mining; Data Mining and Knowledge Discovery; Database Management; Databricks; Health care; Heart rate; Information Storage and Retrieval; Kmeans; Machine learning; Mathematical Applications in Computer Science; Miscarriage; Mobile computing; Networks; Predictions; Predictive analytics; Pregnancy; Real time; Risk analysis; Sensors; Smartphones; Spark; Temperature sensors; Wearable technology; Kmeans; Big data; Spark; Databricks; Data mining; Predictive analytics
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-019-0227-y

Background Mobile phone and sensors have become very useful to understand and analyze human lifestyle because of the huge amount of data they can collect every second. This triggered the idea of combining benefits and advantages of reality mining, machine learning and big data predictive analytics tools, applied to smartphones/sensors real time. The main goal of our study is to build a system that interacts with mobile phones and wearable healthcare sensors to predict patterns. Methods Wearable healthcare sensors (heart rate sensor, temperature sensor and activity sensor) and mobile phone are used for gathering real time data. All sensors are managed using IoT systems; we used Arduino for collecting data from health sensors and Raspberry Pi 3 for programming and processing. Kmeans clustering algorithm is used for patterns prediction and predicted clusters (partitions) are transmitted to the user in his front-end interface in the mobile application. Real world data and clustering validation statistics (Elbow method and Silhouette method) are used to validate the proposed system and assess its performance and effectiveness. All data management and processing tasks are conducted over Apache Spark Databricks. Results This system relies on real time gathered data and can be applied to any prediction case making use of sensors and mobile generated data. As a proof of concept, we worked on predicting miscarriages to help pregnant women make quick decisions in case of miscarriage or probable miscarriage by creating a real time system prediction of miscarriage using wearable healthcare sensors, mobile tools, data mining algorithms and big data technologies. 9 risk factors contribute vastly in prediction, the Elbow method asserts that the optimal number of cluster is 2 and we achieve a higher value (0, 95) of Silhouette width that validates the good matching between clusters and observations. K-means algorithm gives good results in clustering the data.