International Journal of Modern Education and Computer Science (IJMECS)
IJMECS Vol. 5, No. 11, 8 Nov. 2013
Cover page and Table of Contents: PDF (size: 276KB)
Evaluation of Ensemble Classifiers for Handwriting Recognition
Full Text (PDF, 276KB), PP.11-20
Views: 0 Downloads: 0
Author(s)
M.Govindarajan
1,*
Index Terms
Data Mining, Ensemble, Handwriting Recognition, Radial Basis Function, Support Vector Machine, Accuracy.
Abstract
One of the major developments in machine learning in the past decade is the ensemble method, which finds highly accurate classifier by combining many moderately accurate component classifiers. In this research work, new ensemble classification methods are proposed for homogeneous ensemble classifiers using bagging and heterogeneous ensemble classifiers using arcing classifier and their performances are analyzed in terms of accuracy. A Classifier ensemble is designed using Radial Basis Function (RBF) and Support Vector Machine (SVM) as base classifiers. The feasibility and the benefits of the proposed approaches are demonstrated by the means of real and benchmark data sets of recognizing totally unconstrained handwritten numerals. The main originality of the proposed approach is based on three main parts: preprocessing phase, classification phase and combining phase. A wide range of comparative experiments are conducted for real and benchmark data sets of recognizing totally unconstrained handwritten numerals. The accuracy of base classifiers is compared with homogeneous and heterogeneous models for data mining problem. The proposed ensemble methods provide significant improvement of accuracy compared to individual classifiers and also heterogeneous models exhibit better results than homogeneous models for real and benchmark data sets of recognizing totally unconstrained handwritten numerals.
Cite This Paper
M.Govindarajan, "Evaluation of Ensemble Classifiers for Handwriting Recognition", International Journal of Modern Education and Computer Science (IJMECS), vol.5, no.11, pp.11-20, 2013. DOI:10.5815/ijmecs.2013.11.02
Reference
[1]A. Amin, H. B. Al-Sadoun, and S. Fischer, “Hand-printed Arabic Character Recognition System Using An Artificial Network”, Pattern Recognition Vol. 29, No. 4, 1996: 663-675.
[2]Amritha Sampath, Tripti C, Govindaru V, "Freeman code based online handwritten character recognition for Malayalam using backpropagation neural networks", International journal on Advanced computing, Vol. 3, No. 4, 2012: 51 – 58.
[3]Breiman. L, Bias, Variance, and Arcing Classifiers”, Technical Report 460, Department of Statistics, University of California, Berkeley, CA, 1996.
[4]Breiman, L. Bagging predictors. Machine Learning, 24(2):1996a:123–140.
[5]C. J. C. Burges and B. Scholkopf, “Improving the Accuracy and Speed of Support vector Learning Machine”, Advanced in Neural Information Processing Systems 9, MIT Press, Cambridge, MA, 1997: 375-381.
[6]Burges, C. J. C. “A tutorial on support vector machines for pattern recognition”, Data Mining and Knowledge Discovery, 2(2):1998:121-167.
[7]J. Cai, M. Ahmadi, and M. Shridhar, “Recognition of Handwritten Numerals with Multiple Feature and Multi-stage Classifier”, Pattern Recognition, VOL. 28, No. 2, 1995:153-160.
[8]Cherkassky, V. and Mulier, F. “Learning from Data - Concepts, Theory and Methods”, John Wiley & Sons, New York, 1998.
[9]J. X. Dong, A. Krzyzak, and C.Y. Suen, “Fast SVM Training Algorithm with Decomposition on Very Large Datasets”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, No. 4, 2005: 603-618.
[10]Freund, Y. and Schapire, R. “A decision-theoretic generalization of on-line learning and an application to boosting”, In proceedings of the Second European Conference on Computational Learning Theory, 1995: 23-37.
[11]Freund, Y. and Schapire, R. “Experiments with a new boosting algorithm”, In Proceedings of the Thirteenth International Conference on Machine Learning, 1996:148-156 Bari, Italy.
[12]M.Govindarajan, RM.Chandrasekaran, “Intrusion Detection using an Ensemble of Classification Methods”, In Proceedings of International Conference on Machine Learning and Data Analysis, San Francisco, U.S.A, 24-26 October, 2012, pages 459-464.
[13]Haykin, S. “Neural networks: a comprehensive foundation” (second ed.), New Jersey: Prentice Hall, 1999.
[14]T.K.Ho, J.J.Hull, and S.N.Srihari, “Combination of Structural Classifiers”, in Proc. IAPR Workshop Syntatic and Structural Pattern Recog., 1990: 123-137.
[15]Y. S. Huang and C. Y. Suen, “An Optimal Method of Combining Multiple Classifiers for Unconstrained Handwritten Numeral Recognition”, Proceedings of 3rd International Workshop on Frontiers in Handwriting Recognition, 1993.
[16]Y. S. Huang and C. Y. Suen, “A Method of Combining Experts for the Recognition of Unconstrained Handwritten Numerals”, IEEE Transactions on PAMI, Vol. 17, No. 1, 1995: 90-94.
[17]Jiawei Han, Micheline Kamber, “Data Mining – Concepts and Techniques”, Elsevier Publications, 2003.
[18]Kohavi, R. “A study of cross-validation and bootstrap for accuracy estimation and model selection”, Proceedings of International Joint Conference on Artificial Intelligence, 1995:1137–1143.
[19]U. Krebel, “Pairwise Classification and Support Vector Machines, Advances in Kernel Methods: Support Vector Learning”, MIT Press, Cambridge, MA, 1999: 255-268.
[20]L. Lam and C. Y. Suen, “Optimal Combinations of Pattern Classifiers”, Pattern Recognition Letters, Vol. 16, No. 9, 1995: 945-954.
[21]Moncef Charfi, Monji Kherallah, Abdelkarim El Baati, Adel M. Alimi, "A New Approach for Arabic Handwritten Postal Addresses Recognition", International Journal of Advanced Computer Science and Applications, Vol. 3, No. 3, 2012: 1-7.
[22]Muhammad Naeem Ayyaz, Imran Javed, Waqar Mahmood, "Handwritten Character Recognition Using Multiclass SVM Classification with Hybrid Feature Extraction", Pakistan journal of Engineering and Application Science, Vol. 10, 2012: 57-67.
[23]Oliver Buchtala, Manuel Klimek, and Bernhard Sick, Member, IEEE, “Evolutionary Optimization of Radial Basis Function Classifiers for Data Mining Applications”, IEEE Transactions on systems, man, and cybernetics—part b: cybernetics, vol. 35, no. 5, 2005.
[24]Renata F. P. Neves, Alberto N. G. Lopes Filho, Carlos A.B.Mello, CleberZanchettin, “A SVM Based Off-Line Handwritten Digit Recognizer”, International conference on Systems, Man and Cybernetics, IEEE Xplore, 2011: 510-515, Brazil.
[25]D. C. Shubhangi and P. S. Hiremath, “Handwritten English character and digit recognition using multiclass SVM classifier and using structural micro features,” International Journal of Recent Trends in Engineering, vol. 2, no. 2 2009.
[26]C.Y.Suen, C.Nadal, T.A.Mai, R.Legault, and L.Lam, “Recognition of totally unconstrained handwritten numerals based on the concept of multiple experts,” Frontiers in Handwriting Recognition , C.Y.Suen, Ed., IN Proc.Int.Workshop on Frontiers in Handwriting Recognition, Montreal, Canada, Apr. 2-3, 1990” 131-143.
[27]C. Y. Suen, C. Nadal, R. Legault, T. A. Mai, and L. Lam, (1992), “Computer recognition of unconstrained handwritten numerals,” Proc. IEEE, vol. 80, 1992: 1162–1180.
[28]Tsai, C. F., Lu, Y.F. "Customer Churn Prediction by Hybrid Neural Network", Expert Systems with Application (39): 2009: 12547-12553.
[29]Vanajakshi, L. and Rilett, L.R. “A Comparison of the Performance of Artificial Neural Network and Support Vector Machines for the Prediction of Traffic Speed”, IEEE Intelligent Vehicles Symposium, University of Parma, Parma, Italy: IEEE: 2004:194-199.
[30]Vapnik, V. Statistical learning theory, New York, John Wiley & Sons, 1998.
[31]Wang, C.H, and Srihari, S.N. “A framework for object recognition in a visually complex environment and its applications to locating address blocks on mail pieces”, Int J Computer Vision 2, 125, 1998.
[32]L. Xu, A. Krzyzak, and C. Y. Suen, “Methods of Combining Multiple Classifiers and Their Applications to Handwritten Recognition”, IEEE Transactions on Systems, Man, Cybernetics, Vol. 22, No. 3, 1992: 418-435.