DART: Fast and Efficient Distributed Stream Processing Framework for Internet of Things

• Published in: ETRI journal Vol. 39; no. 2; pp. 202 - 212
• Main Authors: Choi, Jang‐Ho, Park, Junyong, Park, Hwin Dol, Min, Ok‐gee
• Format: Journal Article
• Language: English
• Published: Electronics and Telecommunications Research Institute (ETRI) 01.04.2017; 한국전자통신연구원
• Subjects: Distributed system; Event‐based processing; Internet of things; Stream processing; 전자/정보통신공학
• ISSN: 1225-6463; 2233-7326
• DOI: 10.4218/etrij.17.2816.0109

With the advent of the Internet‐of‐Things paradigm, the amount of data production has grown exponentially and the user demand for responsive consumption of data has increased significantly. Herein, we present DART, a fast and lightweight stream processing framework for the IoT environment. Because the DART framework targets a geospatially distributed environment of heterogeneous devices, the framework provides (1) an end‐user tool for device registration and application authoring, (2) automatic worker node monitoring and task allocations, and (3) runtime management of user applications with fault tolerance. To maximize performance, the DART framework adopts an actor model in which applications are segmented into microtasks and assigned to an actor following a single responsibility. To prove the feasibility of the proposed framework, we implemented the DART system. We also conducted experiments to show that the system can significantly reduce computing burdens and alleviate network load by utilizing the idle resources of intermediate edge devices.