Parallel big data processing system for security monitoring in Internet of Things networks

Igor V. Kotenko, Igor Saenko, Alexey Kushnerevich
2017 Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications  
Nowadays, the Internet of Things (IoT) networks are increasingly used in many areas. At the same time, the approach connected with the implementation of the network security monitoring system is of particular relevance for the protection of IoT networks from threats. Due to the peculiarities for construction and operation of IoT networks, the use of traditional protection systems for IoT is difficult or impossible. One of such features is the need to analyze very large amounts of data in real
more » ... me and with minimal computational cost. Given the limited computing capabilities of IoT networks, we propose the architecture of a big data distributed parallel processing system based on Hadoop and Spark software platforms. The issues related to the implementation of this system and its main components are also considered. The results of an experimental evaluation of the system performance are discussed. They confirm the conclusion about its high efficiency. A comparative evaluation of the implemented systems on Hadoop and Spark platforms is conducted. 60 Parallel big data processing system Kotenko, Saenko, and Kushnerevich traditional methods and means of information security are not effective enough in IoT networks. This is due to the low computing power of IoT network resources and the large number of different types of used communication networks. For these reasons, a new approach is of particular relevance for the security of IoT networks. This approach is associated with the creation and application of security information and event management (SIEM) systems [4, 5, 6] . SIEM systems monitor network security. The monitoring consists of collecting data about security events from remote devices, information sensors and network elements and their preliminary processing. However, a large number of data sources types that are used for network security monitoring and high intensity of event streams lead to the need of developing new solutions for processing of Big Data. One such solution is the approaches proposed in this paper to develop a system for parallel processing of security data intended for implementation in IoT networks. The developed parallel data processing system has the following features determining the theoretical and practical significance of the paper. First, due to the use of Complex Event Processing (CEP) technology, the system implements basic real-time pre-processing functions, which are data normalization, data filtering, data aggregation and data correlation. Secondly, the results of preliminary processing are provided by visual representation (visualization). For this purpose, not only standard, but also specially designed visualization models are used. Thirdly, the system operates under the conditions of inherent computational limitations of the IoT network elements. In this case, the basis for building a parallel security data processing system is the Hadoop open source software environment. As it is known, Hadoop is currently the most widespread and rather flexible platform, allowing to create parallel processing systems [7, 8, 9] . In addition, a Spark distributed data processing environment was used. This allowed us to compare the efficiency of the Hadoop and Spark platforms to develop a system for parallel processing. Thus, the main goal of the work is to develop and study architectural and system solutions aimed at creating a parallel processing system for IoT network monitoring, as well as comparing the performance of Hadoop and Spark parallel processing platforms to be used to build this system.
doi:10.22667/jowua.2017.12.31.060 dblp:journals/jowua/KotenkoSK17 fatcat:jxuz4kfxxrhonc4b4pbrqwtmxy