Ayman E. Khedr; S.E.Salama; Nagwa Yaseen

Predicting Stock Market Behavior using Data Mining Technique and News Sentiment Analysis

Full Text (PDF, 662KB), PP.22-30

Views: 0 Downloads: 0

Author(s)

Ayman E. Khedr ^1,* S.E.Salama ^2,3 Nagwa Yaseen ²

1. Faculty of Computers and Information Technology, Future University in Egypt, Information Systems Department, Cairo, Egypt

2. Faculty of Computers and Information, Helwan University, Information Systems Department Cairo, Egypt

3. Faculty of Computers and Information Technology, King Abduaziz University, Jeddah, Kingdom of Saudi

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2017.07.03

Received: 20 Nov. 2016 / Revised: 19 Feb. 2017 / Accepted: 20 Apr. 2017 / Published: 8 Jul. 2017

Index Terms

Data Mining, Stock Market, sentiment analysis, Text Mining, news sentiment analysis

Abstract

Stock market prediction has become an attractive investigation topic due to its important role in economy and beneficial offers. There is an imminent need to uncover the stock market future behavior in order to avoid investment risks. The large amount of data generated by the stock market is considered a treasure of knowledge for investors. This study aims at constructing an effective model to predict stock market future trends with small error ratio and improve the accuracy of prediction. This prediction model is based on sentiment analysis of financial news and historical stock market prices. This model provides better accuracy results than all previous studies by considering multiple types of news related to market and company with historical stock prices. A dataset containing stock prices from three companies is used. The first step is to analyze news sentiment to get the text polarity using naïve Bayes algorithm. This step achieved prediction accuracy results ranging from 72.73% to 86.21%. The second step combines news polarities and historical stock prices together to predict future stock prices. This improved the prediction accuracy up to 89.80%.

Cite This Paper

Ayman E. Khedr, S.E.Salama, Nagwa Yaseen,"Predicting Stock Market Behavior using Data Mining Technique and News Sentiment Analysis", International Journal of Intelligent Systems and Applications(IJISA), Vol.9, No.7, pp.22-30, 2017. DOI:10.5815/ijisa.2017.07.03

Reference

[1]B. O. Wyss, “Fundamentals of the stock market,” p. 245, 2000.
[2]M. K. Jiawei Han, Data MiningConcepts and Techniques, Second Edi. Urbana-Champaign, 2006.
[3]K. Tan , steinbach, Introduction to data mining. 2006.
[4]W. Walter, K. Ho, W. R. Liu, and K. Tracy, “The relation between news events and stock price jump : an analysis based on neural network,” 20th Int. Congr. Model. Simulation, Adelaide, Aust. 1–6 December 2013 www.mssanz.org.au/modsim2013, no. December, pp. 1–6, 2013.
[5]A. Søgaard, “Sentiment analysis and opinion mining,” … Lang. Comput. Group, Microsoft Res. Asia …, no. May, 2013.
[6]K. J. C. Sahil Zubair, “Extracting News Sentiment and Establishing its Relationship with the S & P 500 Index,” 48th Hawaii Int. Conf. Syst. Sci. Extr., 2015.
[7]C. Paper, “Preprocessing Techniques for Text Mining Preprocessing Techniques for Text Mining,” J. Emerg. Technol. Web Intell., no. October 2014, 2016.
[8]pasi tapanainen gregory grefenstette, “™ what is a word,what is a sentence?problem of tokanization,” maylan Fr., p. 9, 1994.
[9]H. D. Hana Alostad, “Directional Prediction of Stock Prices using Breaking News on Twitter,” IEEE/WIC/ACM Int. Conf. Web Intell. Intell. Agent Technol., pp. 0–7, 2015.
[10]M. F. Patrick Uhr, Johannes Zenkert, “Sentiment Analysis in Financial Markets,” IEEE Int. Conf. Syst. Man, Cybern., pp. 912–917, 2014.
[11]P. M. Hoang Thanh, “Stock Market Trend Prediction Based on Text Mining of Corporate Web and Time Series Data,” J. Adv. Comput. Intell. Intell. Informatics, vol. 18, no. 1, 2014.
[12]S. M. Price, J. Shriwas, and S. Farzana, “Using Text Mining and Rule Based Technique for Prediction of,” Int. J. Emerg. Technol. Adv. Eng., vol. 4, no. 1, 2014.
[13]L. I. Bing and C. Ou, “Public Sentiment Analysis in Twitter Data for Prediction of A Company ’ s Stock Price Movements,” IEEE 11th Int. Conf. E-bus. Eng. Public, 2014.
[14]B. D. T. Yahya Eru Cakra, “Stock Price Prediction using Linear Regression based on Sentiment Analysis,” Int. Conf. Adv. Comput. Sci. Inf. Syst., pp. 147–154, 2015.
[15]Y. Shynkevichl, T. M. Mcginnityl, S. Colemanl, and A. Belatrechel, “Stock Price Prediction based on Stock-Specific and Sub-Industry-Specific News Articles,” 2015.
[16]S. S. Umbarkar and P. S. S. Nandgaonkar, “Using Association Rule Mining : Stock Market Events Prediction from Financial News,” vol. 4, no. 6, pp. 1958–1963, 2015.
[17]R. Desai, “Stock Market Prediction Using Data Mining 1,” vol. 2, no. 2, pp. 2780–2784, 2014.
[18]I. Journal, O. F. Social, and H. Studies, “TIME SERIES ANALYSIS ON STOCK MARKET FOR TEXT MINING,” vol. 6, no. 1, pp. 69–91, 2014.
[19]Y. Kim, S. R. Jeong, and I. Ghani, “Text Opinion Mining to Analyze News for Stock Market Prediction,” Int. J. Adv. Soft Comput. Its Appl., vol. 6, no. 1, pp. 1–13, 2014.
[20]S. S. Abdullah, M. S. Rahaman, and M. S. Rahman, “Analysis of stock market using text mining and natural language processing,” 2013 Int. Conf. Informatics, Electron. Vis., pp. 1–6, 2013.
[21]U. States and E. Commission, “SECURITIES AND EXCHANGE COMMISSION THE ‘TRANSITION REPORT PURSUANT TO SECTION 13 OR 15 ( d ) OF THE SECURITIES,” vol. 302, 2014.
[22]N. L. More, C. T. Any, and O. U. S. Equities, “About NASDAQ.”
[23]D. Lyon and B. Cedex, “N-grams based feature selection and text representation for Chinese Text Classification ZhihuaWEI,” Int. J. Comput. Intell. Syst., vol. 2, no. 4, pp. 365–374, 2009.
[24]salton and Buckley, “Term Weighting Approaches in Automatic Text Retrieval,” Inf. Process. Manag., vol. 24(5), p. 513–523., 1988.
[25]V. Kotu and B. Deshpande, Predictive Analytics and Data Mining. 2015.
[26]M. Mittermayer, “Forecasting Intraday Stock Price Trends with Text Mining Techniques,” Proc. 37th Hawaii Int. Conf. Syst. Sci. - 2004, vol. 0, no. C, pp. 1–10, 2004.
[27]S. B. Imandoust and M. Bolandraftar, “Application of K-Nearest Neighbor ( KNN ) Approach for Predicting Economic Events : Theoretical Background,” vol. 3, no. 5, pp. 605–610, 2013.
[28]Mr. B. Narendra and Mr. K. Uday Sai et al., “Sentiment Analysis on Movie Reviews : A Comparative Study of Machine Learning Algorithms and Open Source Technologies,” IJISA, no. August, pp. 66–70, 2016..
[29]P. A. Idowu, C. Osakwe, A. A. Kayode, and E. R. Adagunodo, “Prediction of Stock Market in Nigeria Using Artificial Neural Network,” IJISA, no. October, pp. 68–74, 2012.
[30]N. and K. J. Navale, “Prediction of Stock Market using Data Mining and Artificial Intelligence,” Int. J. Comput. Appl., vol. 134, no. 12, pp. 9–11, 2016.

International Journal of Intelligent Systems and Applications (IJISA)