Ayodeji O. J. Ibitoye; Olufade F.W. Onifade

Utilizing RoBERTa Model for Churn Prediction through Clustered Contextual Conversation Opinion Mining

Full Text (PDF, 537KB), PP.1-8

Views: 0 Downloads: 0

Author(s)

Ayodeji O. J. Ibitoye ^1,* Olufade F.W. Onifade ²

1. School of Computing and Mathematical Science, Faculty of Engineering and Science, University of Greenwich, SE10 9LS, London, United Kingdom

2. Department of Computer Science, Faculty of Science, University of Ibadan, Nigeria

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2023.06.01

Received: 12 Aug. 2023 / Revised: 24 Sep. 2023 / Accepted: 20 Oct. 2023 / Published: 8 Dec. 2023

Index Terms

Churn Prediction, Opinion Mining, Roberta Model, Customer Relationship Management, Decision Support

Abstract

In computational study and automatic recognition of opinions in free texts, certain words in sentences are used to decide its sentiments. While analysing each customer’s opinion per time in churn management will be effective for personalised recommendations. Oftentimes, the opinion is not sufficient for contextualised content mining. While personalised recommendations are time consuming, it also does not provide complete picture of an overall sentiment in the business community of customers. To help businesses identify widespread issues affecting a large segment of their customers towards engendering patterns and trends of different customer churn behaviour, here, we developed a clustered contextualised conversation as opinions set for integration with Roberta Model. The developed churn behavioural opinion clusters disambiguated short messages while charactering contents collectively based on context beyond keyword-based sentiment matching for effective mining. Based on the predicted opinion threshold, customer churn category for group-based personalised decision support was generated, with matching concepts. The baseline RoBERTa model on the contextually clustered opinions, trained with a batch size of 16, a learning rate of 2e-5, over 8 epochs, using a maximum sequence length of 128 and standard hyperparameters, achieved an accuracy of 92%, Precision of 88%, Recall of 86% and F1 score of 84% over a test set of 30%.

Cite This Paper

Ayodeji O. J. Ibitoye, Olufade F.W. Onifade, "Utilizing RoBERTa Model for Churn Prediction through Clustered Contextual Conversation Opinion Mining", International Journal of Intelligent Systems and Applications(IJISA), Vol.15, No.6, pp.1-8, 2023. DOI:10.5815/ijisa.2023.06.01

Reference

[1]Evans, Dave. Social media marketing: the next generation of business engagement. John Wiley & Sons, 2010.
[2]Bashar, Abu, Irshad Ahmad, and Mohammad Wasiq. "Effectiveness of social media as a marketing tool: An empirical study." International Journal of Marketing, Financial Services & Management Research 1, no. 11 88-99: (2012)
[3]Mishra, Abinash, and U. Srinivasulu Reddy. "A comparative study of customer churn prediction in telecom industry using ensemble-based classifiers." In 2017 International conference on inventive computing and informatics (ICICI), pp. 721-725. IEEE, 2017.
[4]Bi, Qingqing. "Cultivating loyal customers through online customer communities: A psychological contract perspective." Journal of Business Research 103 (2019): 34-44.
[5]Mewari, Ritu, Ajit Singh, and Akash Srivastava. "Opinion mining techniques on social media data." International Journal of Computer Applications 118, no. 6 (2015).
[6]Tayal, Devendra Kumar, Sumit Kumar Yadav, and Divya Arora. "Personalized ranking of products using aspect-based sentiment analysis and Plithogenic sets." Multimedia Tools and Applications 82, no. 1 (2023): 1261-1287.
[7]Madhuri, D. and Prasad, R., 2020. A ML and NLP based Framework for Sentiment Analysis on Bigdata. International Journal of Recent Technology and Engineering, 8(5), pp.189-200.
[8]Ayub, Nafees, Muhammad Ramzan Talib, Muhammad Kashif Hanif, and Muhammad Awais. "Aspect Extraction Approach for Sentiment Analysis Using Keywords." Computers, Materials & Continua 74, no. 3 (2023).
[9]Al-Otaibi, Shaha, Allulo Alnassar, Asma Alshahrani, Amany Al-Mubarak, Sara Albugami, Nada Almutiri, and Aisha Albugami. "Customer satisfaction measurement using sentiment analysis." International Journal of Advanced Computer Science and Applications 9, no. 2 (2018).
[10]A. Amin, F. Al-Obeidat, B. Shah, A. Adnan, J. Loo and S. Anwar, "Customer churn prediction in telecommunication industry using data certainty," Journal of Business Research, vol. 94, pp. 290-301, 2019.
[11]Umayaparvathi, V., and K. Iyakutti. "A survey on customer churn prediction in telecom industry: Datasets, methods and metrics." International Research Journal of Engineering and Technology (IRJET) 3, no. 04 (2016).
[12]Ullah, Irfan, Basit Raza, Ahmad Kamran Malik, Muhammad Imran, Saif Ul Islam, and Sung Won Kim. "A churn prediction model using random forest: analysis of machine learning techniques for churn prediction and factor identification in telecom sector." IEEE access 7 (2019): 60134-60149.
[13]Lalwani, Praveen, Manas Kumar Mishra, Jasroop Singh Chadha, and Pratyush Sethi. "Customer churn prediction system: a machine learning approach." Computing 1-24, 2022
[14]Lee, S., Kim, J., Park, H. “Opinion Mining for Churn Prediction: A Review of Techniques and Applications”. International Journal of Data Science and Knowledge Engineering, 9(3), 215-230, 2021
[15]Nassirtoussi, Arman Khadjeh, Saeed Aghabozorgi, Teh Ying Wah, and David Chek Ling Ngo. "Text mining for market prediction: A systematic review." Expert Systems with Applications 41, no. 16 (2014): 7653-7670.
[16]Ibitoye, Ayodeji OJ, and Olufade FW Onifade. "Improved customer churn prediction model using word order contextualized semantics on customers’ social opinion." Int J Adv Appl Sci 11, no. 2 (2022): 107-112.
[17]Wang, L., Zhang, Q., Li, X. “Leveraging Social Media Data for Customer Churn Prediction: A Review of Opinion Mining Approaches”. International Journal of Information Management, 42, 131-147. 2018
[18]Gupta, R., Sharma, A., Kumar, V.” Customer Churn Prediction using Opinion Mining and Topic Modeling: A Literature Review”. International Journal of Computational Intelligence and Applications, 18(4), 1950021. 2019
[19]Chen, S., Wang, Y., Liu, H. “A Comparative Review of Opinion Mining Techniques for Customer Churn Prediction.” Journal of Marketing Analytics, 4(2), 89-105, 2017
[20]Jena, M., and Sanghamitra Mohanty. "Contextual opinion mining in online Odia text using support vector machine." Compliance Eng. J 10, no. 7 (2019): 166-169.
[21]Cheng, Li-Chen, and Chi-Lun Huang. "Exploring contextual factors from consumer reviews affecting movie sales: an opinion mining approach." Electronic commerce research 20 (2020): 807-832.
[22]Sundermann, Camila Vaccari, Marcos Aurélio Domingues, Roberta Akemi Sinoara, Ricardo Marcondes Marcacini, and Solange Oliveira Rezende. "Using opinion mining in context-aware recommender systems: A systematic review." Information 10, no. 2 (2019): 42.
[23]Tan, Kian Long, Chin Poo Lee, Kalaiarasi Sonai Muthu Anbananthen, and Kian Ming Lim. "RoBERTa-LSTM: a hybrid model for sentiment analysis with transformer and recurrent neural network." IEEE Access 10 (2022): 21517-21525.
[24]Phan, Minh Hieu, and Philip O. Ogunbona. "Modelling context and syntactical features for aspect-based sentiment analysis." In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 3211-3220. 2020.
[25]AL-Sharuee, Murtadha Talib, Fei Liu, and Mahardhika Pratama. "Sentiment analysis: an automatic contextual analysis and ensemble clustering approach and comparison." Data & Knowledge Engineering 115 (2018): 194-213.
[26]Rizun, Nina, Katarzyna Ossowska, and Yurii Taranenko. "Modeling the customer’s contextual expectations based on latent semantic analysis algorithms." In Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architecture and Technology–ISAT 2017: Part II, pp. 364-373. Springer International Publishing, 2018.

International Journal of Intelligent Systems and Applications (IJISA)