Performance Evaluation of Fake News Detection Models

PDF (743KB), PP.89-100

Views: 0 Downloads: 0

Author(s)

Afeez Ayomide Olagunju 1,* Iyabo Olukemi Awoyelu 1

1. Department of Computer Science and Engineering, Obafemi Awolowo University, Ile-Ife, Nigeria

* Corresponding author.

DOI: https://doi.org/10.5815/ijitcs.2024.06.07

Received: 17 Aug. 2024 / Revised: 20 Sep. 2024 / Accepted: 17 Oct. 2024 / Published: 8 Dec. 2024

Index Terms

Fake News Detection, Ensemble Learning, Deep Learning, Twitter Bot, Social Media Platforms

Abstract

The rapid spread of misinformation on social media platforms, especially Twitter, presents a challenge in the digital age. Traditional fact-checking struggles with the volume and speed of misinformation, while existing detection systems often focus solely on linguistic features, ignoring factors like source credibility, user interactions, and context. Current automated systems also lack the accuracy to differentiate between genuine and fake news, resulting in high rates of false positives and negatives. This study investigates the creation of a Twitter bot for detecting fake news using deep learning methodologies. The research assessed the performance of BERT, CNN, and Bi-LSTM models, along with an ensemble model combining their strengths. The TruthSeeker dataset was used for training and testing. The ensemble model leverages BERT's contextual understanding, CNN’s feature extraction, and Bi-LSTM’s sequence learning to improve detection accuracy. The Twitter bot integrates this ensemble model via the Twitter API for real-time detection of fake news. Results show the ensemble model significantly outperformed individual models and existing systems, achieving an accuracy of 98.24%, recall of 98.14%, precision of 98.42%, and an F1-score of 98.24%. These findings highlight that combining multiple models can offer an effective solution for real-time detection of misinformation, contributing to efforts to combat fake news on social media platforms.

Cite This Paper

Afeez Ayomide Olagunju, Iyabo Olukemi Awoyelu, "Performance Evaluation of Fake News Detection Models", International Journal of Information Technology and Computer Science(IJITCS), Vol.16, No.6, pp.89-100, 2024. DOI:10.5815/ijitcs.2024.06.07

Reference

[1]X. Zhang and A. A. Ghorbani, "An overview of online fake news: Characterization, detection, and discussion," Information Processing and Management, vol. 57, no. 2, pp. 102-125, 2020.
[2]B. Akinyemi, O. Adewusi, and A. Oyebade, "An improved classification model for fake news detection in social media," Int. J. Inf. Technol. Comput. Sci., vol. 12, no. 1, pp. 34-43, 2020.
[3]S. Yang, K. Shu, S. Wang, R. Gu, F. Wu, and H. Liu, "Unsupervised fake news detection on social media: A generative approach," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 5644-5651, July 2019.
[4]V. Verma, M. Rohilla, A. Sharma, and M. Gupta, "Fake news detection on Twitter," in Advances in Data and Information Sciences: Proceedings of ICDIS 2022, Singapore: Springer Nature Singapore, 2022, pp. 141-149.
[5]A. Mitchell, "News on Twitter: Consumed by most users and trusted by many," Pew Research Center, Nov. 15, 2021. [Online]. Available: https://www.pewresearch.org/Journalism/2021/11/15/news-on-twitter-consumed-by-most-users-and-trusted-by-many/. [Accessed: Aug. 22, 2023].
[6]Pew Research Center, "Social media and news fact sheet," Pew Research Center, 2022. [Online]. Available: https://www.pewresearch.org/Journalism/fact-sheet/social-media-and-news-fact-sheet/. [Accessed: Oct. 20, 2023].
[7]S. Kemp, "Digital 2023: Nigeria – Datareportal – Global digital insights," Datareportal, 2023. [Online]. Available: https://datareportal.com/reports/digital-2023-nigeria. [Accessed: Jan. 19, 2024].
[8]R. J. Oentaryo, A. Murdopo, P. K. Prasetyo, and E. P. Lim, "On profiling bots in social media," in Social Informatics: 8th International Conference, SocInfo 2016, Bellevue, WA, USA, November 11-14, 2016, Proceedings, Part I, Springer International Publishing, 2016, pp. 92-109.
[9]T. Lokot and N. Diakopoulos, "News bots: Automating news and information dissemination on Twitter," Digital Journalism, vol. 4, no. 6, pp. 682-699, 2016.
[10]K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, "Fake news detection on social media: A data mining perspective," ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 22-36, 2017.
[11]Y. M. Rocha, G. A. De Moura, G. A. Desidério, C. H. De Oliveira, F. D. Lourenço, and L. D. de Figueiredo Nicolete, "The impact of fake news on social media and its influence on health during the COVID-19 pandemic: A systematic review," Journal of Public Health, pp. 1-10, 2021.
[12]K. Stahl, "Fake news detection in social media," California State University Stanislaus, 2018.
[13]V. L. Rubin, "Deception detection and rumor debunking for social media," in The SAGE Handbook of Social Media Research Methods, London: Sage, 2017, p. 342.
[14]N. I. Saputri, Y. Sibaroni, and S. S. Prasetiyowati, "Covid-19 fake news detection on Twitter based on author credibility using information gain and KNN methods," Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 7, no. 1, pp. 185-192, 2023.
[15]S. Sharma, M. Saraswat, and A. K. Dubey, "Fake news detection on Twitter," International Journal of Web Information Systems, vol. 18, no. 5/6, pp. 388-412, 2022.
[16]E. Amer, K. S. Kwak, and S. El-Sappagh, "Context-based fake news detection model relying on deep learning models," Electronics, vol. 11, no. 8, p. 1255, 2022.
[17]T. Ahmad, M. S. Faisal, A. Rizwan, R. Alkanhel, P. W. Khan, and A. Muthanna, "Efficient fake news detection mechanism using enhanced deep learning model," Applied Sciences, vol. 12, no. 3, pp. 17-43, 2022.
[18]P. K. Varshney, G. K. Wadhwani, and G. M. Upadhyay, "Systematic approach for fake news detection using machine learning," NeuroQuantology, vol. 20, no. 15, pp. 1819-1828, 2022.
[19]W. Y. Wang, "Liar, liar pants on fire: A new benchmark dataset for fake news detection," arXiv preprint arXiv:1705.00648, 2017.
[20]S. Alyoubi, M. Kalkatawi, and F. Abukhodair, "The detection of fake news in Arabic tweets using deep learning," Applied Sciences, vol. 13, no. 14, p. 8209, 2023.
[21]N. Rai, D. Kumar, N. Kaushik, C. Raj, and A. Ali, "Fake news classification using transformer-based enhanced LSTM and BERT," International Journal of Cognitive Computing in Engineering, vol. 3, pp. 98-105, 2022.
[22]Mridha, M. F., Keya, A. J., Hamid, M. A., Monowar, M. M., & Rahman, M. S. (2021). A comprehensive review on fake news detection with deep learning. IEEE access, 9, 156151-156170.
[23]S. Dadkhah, X. Zhang, A. G. Weismann, A. Firouzi, and A. A. Ghorbani, "The largest social media ground-truth dataset for real/fake content: Truthseeker," IEEE Transactions on Computational Social Systems, 2023.
[24]Y. Verma, "Complete guide to bidirectional LSTM (with Python codes)," Analytics India Magazine, 2021. [Online]. Available: https://analyticsindiamag.com/complete-guide-to-bidirectional-lstm-with-python-codes/. [Accessed: Feb. 9, 2023].