Modeling and Simulation of Spark Streaming

Jia-Chun Lin, Ming-Chang Lee, Ingrid Chieh Yu, Einar Broch Johnsen
<span title="">2018</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/6fvrx6ji2nauffetsxyi4abwam" style="color: black;">2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA)</a> </i> &nbsp;
As more and more devices connect to Internet of Things, unbounded streams of data will be generated, which have to be processed "on the fly" in order to trigger automated actions and deliver real-time services. Spark Streaming is a popular realtime stream processing framework. To make efficient use of Spark Streaming and achieve stable stream processing, it requires a careful interplay between different parameter configurations. Mistakes may lead to significant resource overprovisioning and bad
more &raquo; ... performance. To alleviate such issues, this paper develops an executable and configurable model named SSP (stands for Spark Streaming Processing) to model and simulate Spark Streaming. SSP is written in ABS, which is a formal, executable, and object-oriented language for modeling distributed systems by means of concurrent object groups. SSP allows users to rapidly evaluate and compare different parameter configurations without deploying their applications on a cluster/cloud. The simulation results show that SSP is able to mimic Spark Streaming in different scenarios.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/aina.2018.00068">doi:10.1109/aina.2018.00068</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/aina/LinLYJ18.html">dblp:conf/aina/LinLYJ18</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/6edirdluvnde7bvgubytwiievu">fatcat:6edirdluvnde7bvgubytwiievu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200506100851/https://www.duo.uio.no/bitstream/handle/10852/71512/arXiv+version.pdf?sequence=2" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/51/1a/511a3a1f5add75b8e29725c8e2a4307b5d0feb21.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/aina.2018.00068"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>