Edosa Osa; Emmanuel J. Edifon; Solomon Igori

Performance Analysis of Shallow and Deep Learning Classifiers Leveraging the CICIDS 2017 Dataset

PDF (997KB), PP.42-55

Views: 0 Downloads: 0

Author(s)

Edosa Osa ^1,* Emmanuel J. Edifon ¹ Solomon Igori ²

1. Department of Electrical and Electronic Engineering, Faculty of Engineering, University of Benin, P.M.B. 1154, Benin City, Nigeria

2. BBC Dock House, The Quays, Salford M50 2LH, United Kingdom

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2025.02.04

Received: 5 Sep. 2024 / Revised: 27 Oct. 2024 / Accepted: 11 Dec. 2024 / Published: 8 Apr. 2025

Index Terms

CICIDS 2017, Accuracy, Shallow Learning, Deep Learning, Algorithm

Abstract

In order to implement the advantages of machine learning in the cybersecurity ecosystem, various anomaly detection-based models are being developed owing to their ability to flag zero-day attacks over their signature-based counterparts. The development of these anomaly detection-based models depends heavily on the dataset being employed in terms of factors such as wide attack pool or diversity. The CICIDS 2017 stands out as a relevant dataset in this regard. This work involves an analytical comparison of the performances by selected shallow machine learning algorithms as well as a deep learning algorithm leveraging the CICIDS 2017 dataset. The dataset was imported, pre-processed and necessary feature selection and engineering carried out for the shallow learning and deep learning scenarios respectively. Outcomes from the study show that the deep learning model presented the highest performance of all with respect to accuracy score, having percentage value as high as 99.71% but took the longest time to process with 550 seconds. Furthermore, some shallow learning classifiers such as Decision Tree and Random Forest took less processing time (4.567 and 3.95 seconds respectively) but had slightly less accuracy scores than the deep learning model with the CICIDS 2017 dataset. Results from our study show that Deep Neural Network is a viable model for intrusion detection with the CICIDS 2017 dataset. Furthermore, the results of this study are to provide information that may influence choices while developing machine learning based intrusion detection systems with the CICIDS 2017 dataset.

Cite This Paper

Edosa Osa, Emmanuel J. Edifon, Solomon Igori, "Performance Analysis of Shallow and Deep Learning Classifiers Leveraging the CICIDS 2017 Dataset", International Journal of Intelligent Systems and Applications(IJISA), Vol.17, No.2, pp.42-55, 2025. DOI:10.5815/ijisa.2025.02.04

Reference

[1]Lionel Sujay Vailshery, “Number of IoT Connections Worldwide 2022-2033, with Forecasts to 2030”, 2024. Available at: https://www.statista.com/statistics/1183457/iot-connected-devices-worldwide/ Accessed 21 August 2024.
[2]BUSINESS WIRE, “The Growth in Connected IoT Devices is Expected to Generate 79.4ZB of Data in 2025, According to a New IDC Forecast”, 2019. Available at: https://www.businesswire.com/news/home/20190618005012/en/The-Growth-in-Connected-IoT-Devices-is-Expected-to-Generate-79.4ZB-of-Data-in-2025-According-to-a-New-IDC-Forecast Accessed 9 August 2024.
[3]Naushad Alam, Muqeem Ahmed, “Zero-day Network Intrusion Detection using Machine Learning Approach”, 2023. International Journal on Recent and Innovation Trends in Computing and Communication, 11(8s) 194-201. DOI: 10.17762/ijritcc.v11i8s.7190.
[4]Priya Pitre, Arya Gandhi, Vaishnavi Konde, Rahul Adhao, Vinod Pachghare “An Intrusion Detection System for Zero-Day Attacks to Reduce False Positive Rates”, 2022. International Conference for Advancement in Technology (ICONAT), Goa, India, 2022, pp. 1-6, DOI: 10.1109/ICONAT53423.2022.9726105.
[5]Iman Sharafaldin, Amirhossein Gharib, Arash Habibi Lashkari, Ali A. Ghorbani, “Towards a Reliable Intrusion Detection Benchmark Dataset”, 2017. Software Networking, vol. 1, no. 1, pp. 177-200. DOI: 10.13052/jsn2445-9739.2017.009.
[6]Mohammed Al-Garadi, Amr Mohamed, Abdulla K. Al-Ali, Xiaojiang Du, Mohsen Guizani, “A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security”, 2020. IEEE Commun. Surv. Tutor. 22 (3) (2020) 1646–1685, http://dx.doi.org/10.1109/ COMST.2020.2988293.
[7]Fatsuma Jauro, Haruna Chiroma, Abdulsalam Gital, Mubarak Almutairi, Shafi’i Muhammad Abdulhamid, Jemal Abawajy, “Deep Learning Architectures in Emerging Cloud Computing Architectures: Recent Development, Challenges and Next Research Trend”, 2020. Appl. Soft Comput. 96 (2020) 106582, DOI: 10.1016/j.asoc.2020.106582.
[8]Amirhossein Gharib, Iman Sharafaldin, Arash Habibi Lashkari, Ali A. Ghorbani, “An Evaluation Framework for Intrusion Detection Dataset”, 2016. In: 2016 International Conference on Information Science and System (ICISS), pp. 1-6. DOI: 10.1109/ICISSEC.2016.7885840.
[9]Ranjit Panigrahi, Samarjeet Borah, “A Detailed Analysis of CICIDS2017 Dataset for Designing Intrusion Detection Systems”, 2018. International Journal of Engineering & Technology, 7 (3.24) (2018) 479-482. DOI:10.14419/ijet. v7i3.24.22797.
[10]Hassan A Afolabi, Abdurazzag A Aburas, “RTL-DL: A HYBRID DEEP LEARNING FRAMEWORK FOR DDOS ATTACK DETECTION IN A BIG DATA ENVIRONMENT”, 2022. International Journal of Computer Networks & Communications (IJCNC) Vol.14, No.6, November 2022.
[11]Ulya Sabeel, Shahram Shah Heydari, Harsh Mohanka, Yasmine Bendhaou, Khalid Elgazzar, Khalil El-Khatib, “Evaluation of Deep Learning in Detecting Unknown Network Attacks”, 2019. In: 2019 international conference on Smart Applications, Communications and Networking, SmartNets. DOI: 10.1109/SmartNets48225.2019.9069788.
[12]Muhammad Asad, Muhammad Asim, Talha Javed, Mirza Beg, Hasan Mujtaba, Sohail Abbas, “Deep-Detect: Detection of Distributed Denial of Service Attacks Using Deep Learning”, 2020. Computer Journal. 63(7):983–994. DOI: 10.1093/comjnl/bxz064.
[13]Manimurugan Shanmuganathan, Saad Almutairi, Majed Aborokbah, Naveen Chilamkurti, Subra Ganesan, Rizwan Patan, “Effective Attack Detection in Internet of Medical Things Smart Environment Using a Deep Belief Neural Network”, 2020. IEEE Access, Page(s): 77396-77404, 06 April 2020. DOI: 10.1109/ACCESS.2020.2986013.
[14]Alex I. Getman, Maxim Nikolaevich Goryunov, Andrey Georgievich Matskevich, Dmitry A. Rybolovlev, Anastasiya Nikolskaya, “Deep Learning Applications for Intrusion Detection in Network Traffic”, 2023. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2023;35(4):65-92. (In Russ.). DOI: 10.15514/ISPRAS-2023-35(4)-3.
[15]Maxim Nikolaevich Goryunov, Andrey Georgievich Matskevich, Dmitry A. Rybolovlev, “Synthesis of a Machine Learning Model for Detecting Computer Attacks Based on the CICIDS2017 Dataset”, 2020. Trudy ISP RAN/Proc. ISP RAS, vol. 32, issue 5, pp. 81-94 (in Russian). DOI: 10.15514/ISPRAS–2020–32(5)–6.
[16]Othmane Belarbi, Aftab Khan, Pietro Carnelli, Theodoros Spyridopoulos, “An Intrusion Detection System Based on Deep Belief Networks”, 2022. arXiv:2207.02117v1 [cs.CR] 5 Jul 2022. DOI: 10.48550/arXiv.2207.02117.
[17]Priyanka, Dharmender Kumar, “Decision Tree Classifier: A Detailed Survey”, 2020. International Journal of Information and Decision Sciences, Inderscience Enterprises Ltd, 12(3), 246–269. DOI:10.1504/ijids.2020.10029122.
[18]Woosub Jung, Hongyang Zhao, Minglong Sun, Gang Zhou, “IoT botnet detection via power consumption modeling”, 2020. Smart Health, 15(3), 100103. DOI: 10.1016/j.smhl.2019.100103.
[19]Salwa Alem, David Espes, Laurent Nana, Eric Martin, Florent De Lamotte, “A Novel Bi-Anomaly-Based Intrusion Detection System Approach for Industry 4.0. Future Generation Computer Systems”, 2023. Future Generation Computer Systems 145(4):267-283. DOI: 10.1016/j.future.2023.03.024.
[20]Zina Chkirbene, Aiman Erbad, Ridha Hamila, Amr Mohamed, Mohsen Guizani, And Mounir Hamdi, “TIDCS: A Dynamic Intrusion Detection and Classification System Based Feature Selection”, 2020. IEEE Access pp (99):1-1 DOI: 10.1109/ACCESS.2020.2994931.
[21]James Dzisi Gadze, Akua Acheampomaa Bamfo-Asante, Justice Owusu Agyemang, Henry Nunoo-Mensah, Kwasi Adu-Boahen Opare, “An Investigation into the Application of Deep Learning in the Detection and Mitigation of DDOS attack on SDN Controllers”, 2021. Technologies, 9(1), 14, DOI: 10.3390/technologies9010014.
[22]Ziadoon Kamil Maseer, Yusof Robiah, Nazrulazhar Bahaman, Salama A. Mostafa, Cik Feresa Mohd Foozy, “Benchmarking of Machine Learning for Anomaly Based Intrusion Detection Systems in the CICIDS2017 Dataset”, 2021. IEEE Access, 9, 22351–22370. DOI: 10.1109/ACCESS. 2021.3056614.
[23]Yanyan Zhang, Xiangjin Ran, “A Step-Based Deep Learning Approach for Network Intrusion Detection”, 2021. Computer Modeling in Engineering and Science, 128(3), 1231–1245. DOI: 10.32604/cmes.2021.016866.
[24]Lirim Ashiku, Cihan Dagli, “Network Intrusion Detection System using Deep Learning”, 2021. Procedia Computer Science, 185, 239–247, DOI: 10.1016/j.procs.2021.05.025.
[25]Chaofei Tang, Nurbol Luktarhan, Yuxin Zhao, “SAAE-DNN: Deep Learning Method on Intrusion Detection”, 2020. Symmetry 12(10):1695, pp 1-20, DOI: 10.3390/sym12101695.
[26]Daekyeong Park, Sangsoo Kim, Hyukjin Kwon, Dongil Shin, Dongkyoo ShinPark, “Host-Based Intrusion Detection Model Using Siamese Network”, 2021. IEEE Access, 9, 76614–76623. DOI: 10.1109/ACCESS.2021.3082160.
[27]Shah Md. Istiaque, Asif Iqbal Khan, Zaber Al Hassan, and Sajjad Waheed, “Performance Evaluation of a Smart Intrusion Detection System (IDS) Model”, 2021. European Journal of Engineering and Technology Research, 6(2), 148–152, DOI: 10. 24018/ejers.2021.6.2.2371.
[28]Yanfang Fu, Yishuai Du, Zijian Cao, Qiang Li, Wei Xiang, “A Deep Learning Model for Network Intrusion Detection with Imbalanced Data”, 2022. Electronics, 11(6), 898, DOI: 10.3390/electronics11060898.
[29]Yifan Tang, Lize Gu, Leiting Wang, “Deep Stacking Network for Intrusion Detection” 2022. Sensors, 22(1), 25. DOI: 10.3390/s22010025.
[30]Safi Ullah, Muazzam A. Khan, Jawad Ahmad, Sajjad Shaukat Jamal, Zil E Huma, Muhammad Tahir Hassan, Nikolaos Pitropakis, Arshad, William J. Buchanan, “HDL-IDS: A Hybrid Deep Learning Architecture for Intrusion Detection in The Internet of Vehicles” 2022. Sensors, 22(4), 1340, DOI: 10. 3390/s22041340.
[31]Taehoon Kim, Wooguil Pak, “Robust Network Intrusion Detection System Based on Machine-Learning with Early Classification” 2022. IEEE Access, 10, 10754–10767. DOI: 10.1109/ ACCESS.2022.3145002.
[32]Yenigün, O, “Handling Class Imbalance in Machine Learning”, 2023. Available at: http://medium.com/@okanyenigun. Accessed 5 October 2023.
[33]Pandala, S, “Lazy Predict”, 2022. Available at: https://github.com/shankarpandala/lazypredict/blob/dev/README.md. Accessed 1 October 2024.

International Journal of Intelligent Systems and Applications (IJISA)