Online Decision-Making Using Edge Resources for Content-Driven Stream Processing

• Published in: 2017 IEEE 13th International Conference on e-Science (e-Science) pp. 384 - 392
• Main Authors: Renart, Eduard, Balouek-Thomert, Daniel, Xuan Hu, Jie Gong, Parashar, Manish
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2017
• Subjects: Cloud computing; Engines; Image edge detection; Laser radar; Programming; Runtime; Storms
• DOI: 10.1109/eScience.2017.52

The Internet of Things (IoT) describes the emerging paradigm that connects sensors, often located at the edge of the network, to stream processing engines located at the core of the network to enable online data-driven monitoring, management, and control. As IoT applications require increasing volumes of streaming data to be processed by complex workflows in a timely manner, it is becoming important to also leverage resources closer to the edge. Furthermore, the topology of these workflows and where theyare executed is determined not only by application objectives and available resources, but also by the content of the data streams, however, current stream processing engines do not provide this flexibility. In this paper, we present a programming framework that enables applications to specify data-driven, location- and resource-aware processing of data streams. Specifically, it provides abstractions for specifying where and how a data stream is processed based on its content, spatial and temporal characteristics. We also present an implementation of the framework using an event-driven runtime, where events are associatively described. Finally, we demonstrate the effectiveness of the solution by an evaluation of scalability and performance using a disaster response application usecase.