Applying spark based machine learning model on streaming big data for health status prediction

• Published in: Computers & electrical engineering Vol. 65; pp. 393 - 399
• Main Authors: Nair, Lekha R., Shetty, Sujala D., Shetty, Siddhanth D.
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Ltd 01.01.2018; Elsevier BV
• Subjects: Apache spark; Artificial intelligence; Big Data; Big data machine learning; Computer simulation; Data management; Data processing; Health; Health informatics; Machine learning; Mathematical models; Progressions; Real time; Streaming data processing; Tweet processing; Streaming data processing; Apache spark; Big data machine learning; Tweet processing; Health informatics
• ISSN: 0045-7906; 1879-0755
• DOI: 10.1016/j.compeleceng.2017.03.009

•Real-time health status prediction system and the user communicate via Twitter.•Tweet streams are processed and health attributes extracted using Apache Spark.•Machine learning model applied on streaming data to predict health status.•Predicted health status is sent back as a direct message to the user.•Successfully deployed and tested the system in Amazon Elastic Compute Cloud. Machine learning is one of the driving forces of science and commerce, but the proliferation of Big Data demands paradigm shifts from traditional methods in the application of machine learning techniques on this voluminous data having varying velocity. With the availability of large health care datasets and progressions in machine learning techniques, computers are now well equipped in diagnosing many health issues. This work aims at developing a real time remote health status prediction system built around open source Big Data processing engine, the Apache Spark, deployed in the cloud which focus on applying machine learning model on streaming Big Data. In this scalable system, the user tweets his health attributes and the application receives the same in real time, extracts the attributes and applies machine learning model to predict user's health status which is then directly messaged to him/her instantly for taking appropriate action. [Display omitted]