Rakesh K. Lenka; Rabindra K. Barik; Sasmita Panigrahi; Sai S. Panda

An Improved Hybrid Distributed Collaborative Filtering Model for Recommender Engine using Apache Spark

Full Text (PDF, 314KB), PP.74-81

Views: 0 Downloads: 0

Author(s)

Rakesh K. Lenka ^1,* Rabindra K. Barik ² Sasmita Panigrahi ¹ Sai S. Panda ¹

1. Department of Computer Science & Engineering, IIIT Bhubaneswar, India

2. School of Computer Application, KIIT University, India

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2018.07.08

Received: 23 Jun. 2017 / Revised: 2 Aug. 2017 / Accepted: 15 Sep. 2017 / Published: 8 Jul. 2018

Index Terms

Apache Spark, Recommendation Engine, Collaborative Filtering, Machine learning, KMeans, Bisecting KMeans, Bigdata

Abstract

The present scenario there is a serious need of scalability for efficient analytics of big data. In order to achieve this, technology like MapReduce, Pig and HIVE came into action but when the question comes to scalability; Apache Spark maintains a great position far ahead. In this research paper, it has designed and developed an improved hybrid distributed collaborative model for filtering recommender engine. Execution time, scalability and robustness of the engine are the three evaluation parameters; has been considered for this present study. The present work keeps an eye on recommender system built with help of Apache Spark. Apart from this, it has been proposed and implemented the bisecting KMeans clustering algorithms. It has discussed about the comparative analysis between KMeans and Bisecting KMeans clustering algorithms on Apache Spark environment.

Cite This Paper

Rakesh K. Lenka, Rabindra K. Barik, Sasmita Panigrahi, Sai S. Panda, "An Improved Hybrid Distributed Collaborative Model for Filtering Recommender Engine using Apache Spark", International Journal of Intelligent Systems and Applications(IJISA), Vol.10, No.7, pp.74-81, 2018. DOI:10.5815/ijisa.2018.07.08

Reference

[1]Shapira, Bracha, Francesco Ricci, Paul B. Kantor, and LiorRokach. "Recommender Systems Handbook", 2011.
[2]Ricci, Francesco, LiorRokach, and Bracha Shapira. Introduction to recommender systems handbook. Springer US, 2011.
[3]Casey, Walker Evan. "Scalable Collaborative Filtering Recommendation Algorithms on Apache Spark", 2014.
[4]Hashem, Ibrahim Abaker Targio, Ibrar Yaqoob, Nor Badrul Anuar, Salimah Mokhtar, Abdullah Gani, and Samee Ullah Khan. "The rise of “big data” on cloud computing: Review and open research issues." Information Systems, Vol. 47 pp. 98-115, 2015.
[5]Zhao, Zhi-Dan, and Ming-Sheng Shang. "User-based collaborative-filtering recommendation algorithms on hadoop." In Knowledge Discovery and Data Mining, 2010.WKDD'10. Third International Conference on, pp. 478-481. IEEE, 2010.
[6]Mo, Yijun, Jianwen Chen, Xia Xie, Changqing Luo, and Laurence Tianruo Yang. "Cloud-based mobile multimedia recommendation system with user behavior information." IEEE Systems Journal 8, no. 1, pp. 184-193, 2014.
[7]Shang, Yang, Zhiyang Li, WenyuQu, YujieXu, Zining Song, and Xuefei Zhou. "Scalable collaborative filtering recommendation algorithm with MapReduce." In Dependable, Autonomic and Secure Computing (DASC), 2014 IEEE 12th International Conference on, pp. 103-108. IEEE, 2014.
[8]Wang, Chunzhi, Zhou Zheng, and Zhuang Yang. "The research of recommendation system based on Hadoop cloud platform." In Computer Science & Education (ICCSE), 2014 9th International Conference on, pp. 193-196. IEEE, 2014.
[9]Panigrahi, Sasmita, Rakesh Ku Lenka, and Ananya Stitipragyan. "A Hybrid Distributed Collaborative Filtering Recommender Engine Using Apache Spark." Procedia Computer Science, Vol. 83, pp. 1000-1006, 2016.
[10]Bobadilla, Jesús, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. "Recommender systems survey." Knowledge-Based Systems, Vol. 46, pp.109-132, 2013.
[11]Yang, Xiwang, Yang Guo, Yong Liu, and Harald Steck. "A survey of collaborative filtering based social recommender systems." Computer Communications, Vol. 41, pp. 1-10, 2014.
[12]Walunj, Sachin Gulabrao, and Kishor Sadafale. "An online recommendation system for e-commerce based on apache mahout framework." In Proceedings of the 2013 annual conference on Computers and people research, pp. 153-158. ACM, 2013.
[13]Verma, Jai Prakash, Bankim Patel, and Atul Patel. "Big data analysis: recommendation system with Hadoop framework." In Computational Intelligence & Communication Technology (CICT), 2015 IEEE International Conference on, pp. 92-97. IEEE, 2015.
[14]Ciobanu, Alexandru, and Andreas Lommatzsch. "Development of a News Recommender System based on Apache Flink." In Working Notes of the 7th International Conference of the CLEF Initiative, Evora, Portugal. 2016.
[15]Barik, Rabindra K., Harishchandra Dubey, Arun B. Samaddar, Rajan D. Gupta, and Prakash K. Ray. "FogGIS: Fog Computing for Geospatial Big Data Analytics." arXiv preprint arXiv:1701.02601 (2016).
[16]H. Dubey, J. Yang, N. Constant, A. M. Amiri, Q. Yang, andK. Mankodiya, “Fog data: enhancing telehealth big data through fog computing,” in Proceedings of the ASE BigData & SocialInformatics2015. ACM, , pp. 14, 2015.
[17]Han, Zhijie, and Yujie Zhang. "Spark: A Big Data Processing Platform Based on Memory Computing." In Parallel Architectures, Algorithms and Programming (PAAP), 2015 Seventh International Symposium on, pp. 172-176. IEEE, 2015.
[18]Ding, Dongliang, Dongyue Wu, and Fuli Yu. "An overview on cloud computing platform spark for Human Genome mining." In Mechatronics and Automation (ICMA), 2016 IEEE International Conference on, pp. 2605-2610. IEEE, 2016.
[19]Maarala, AlttiIlari, Mika Rautiainen, MiikkaSalmi, Susanna Pirttikangas, and JukkaRiekki. "Low latency analytics for streaming traffic data with Apache Spark." In Big Data (Big Data), 2015 IEEE International Conference on, pp. 2855-2858. IEEE, 2015.
[20]Harper, F. Maxwell, and Joseph A. Konstan. "The movielens datasets: History and context." ACM Transactions on Interactive Intelligent Systems (TiiS), Vol. 5, no. 4, pp.19, 2016.
[21]Domann, Jaschar, Jens Meiners, Lea Helmers, and Andreas Lommatzsch. "Real-Time News Recommendations Using Apache Spark." In Working Notes of the 7th International Conference of the CLEF Initiative, Evora, Portugal. 2016.
[22]Zhao, Zhi-Dan, and Ming-Sheng Shang. "User-based collaborative-filtering recommendation algorithms on hadoop." In Knowledge Discovery and Data Mining, 2010.WKDD'10. Third International Conference on, pp. 478-481. IEEE, 2010.
[23]Wang, Chunzhi, Zhou Zheng, and Zhuang Yang. "The research of recommendation system based on Hadoop cloud platform." In Computer Science & Education (ICCSE), 2014 9th International Conference on, pp. 193-196. IEEE, 2014.
[24]Ciobanu, Alexandru, and Andreas Lommatzsch. "Development of a News Recommender System based on Apache Flink." In Working Notes of the 7th International Conference of the CLEF Initiative, Evora, Portugal. 2016.
[25]Gupta, N., Lenka, R. K., Barik, R. K., & Dubey, H. "FAIR: A Hadoop-based Hybrid Model for Faculty Information Retrieval System." arXiv preprint arXiv:1706.08018 (2017).
[26]Lenka, R. K., Barik, R. K., Gupta, N., Ali, S. M., Rath, A., & Dubey, H. "Comparative analysis of SpatialHadoop and GeoSpark for geospatial big data analytics." Contemporary Computing and Informatics (IC3I), 2016 2nd International Conference on. IEEE, 2016.

International Journal of Intelligent Systems and Applications (IJISA)