Sumaira Kausar; Younus Javed; Samabia Tehsin; Muhammad Riaz

Vision-based Classification of Pakistani Sign Language

Full Text (PDF, 953KB), PP.9-19

Views: 0 Downloads: 0

Author(s)

Sumaira Kausar ^1,* Younus Javed ¹ Samabia Tehsin ² Muhammad Riaz ³

1. National University of Sciences and Technology (NUST), Islamabad, 46000, PAKISTAN

2. Department of Computer Science, Bahria University, Islamabad, 46000, PAKISTAN

3. QEC, NDU, Islamabad, 46000, Pakistan

* Corresponding author.

DOI: https://doi.org/10.5815/ijigsp.2016.02.02

Received: 4 Nov. 2015 / Revised: 8 Dec. 2015 / Accepted: 4 Jan. 2016 / Published: 8 Feb. 2016

Index Terms

Sign language, shape descriptor, invariance, Pakistani Sign language

Abstract

Automated sign language recognition is one of the important areas of computer vision today, because of its applicability in vast fields of life. This paper presents automated recognition of signs taken from Pakistani Sign Language (PSL). The paper presents empirical analysis of two statistical and one transformation based shape descriptors for the recognition of PSL. A purely vision based, efficient, signer independent, multi-aspect invariant method is proposed for the recognition of 44 signs of PSL. The method has proved its worth by utilizing a very small shape descriptor and giving promising results for a reasonable size of sign dictionary. The proposed methodology achieved an accuracy of 92%.

Cite This Paper

Sumaira Kausar, M. Younus Javed, Samabia Tehsin, Muhammad Riaz,"Vision-based Classification of Pakistani Sign Language", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.8, No.2, pp.9-19, 2016. DOI: 10.5815/ijigsp.2016.02.02

Reference

[1]Xing Guo, Zhenyu Lu, Rongbin Xu, Zhengyi Liu , Jianguo Wu, " Big-screen text input system based on gesture recognition", Proceedings of the 2nd International Conference On Systems Engineering and Modeling (ICSEM-13), 2013.

[2]M. Mohandes, M. Deriche, U. Johar, S. Ilyas, "A signer-independent Arabic Sign Language recognition system using face detection, geometric features, and a Hidden Markov Model ",Computers & Electrical Engineering, Volume 38, Issue 2, March 2012, Pages 422–433.

[3]Tolba, M. F, Samir, Ahmed , Abul-Ela, Magdy, "3D Arabic sign language recognition using linear combination of multiple 2D views ", 8th International Conference on Informatics and Systems (INFOS), Cairo, 2012 , 14-16 May 2012, pp: MM-6 - MM-13.

[4]Sumaira Kausar, M. Younus Javed, "A Survey on Sign Language Recognition," 9th International conference on Frontiers of Information Technology, 19-21 December 2011, Islamabad Pakistan, pp.95-98.

[5]M-K. Hu.Visual pattern recognition by moment invariants. IRE Trans. on Information Theory, IT-8:pp. 179-187, 1962.

[6]Wassner. kinect + reseau de neurone = reconnaissance de gestes. http://tinyurl.com/5wbteug, May 2011.

[7]Sumaira Kausar, M. Younus,Javed, Shaleeza,Sohail: "Recognition of gestures in Pakistani sign language using fuzzy classifier" , 8th conference on Signal processing, computational geometry and artificial vision 2008, Rhodes, Greece, Pages 101-105

[8]Aleem Khalid Alvi, M. Yousuf Bin Azhar, Mehmood Usman, Suleman Mumtaz, Sameer Rafiq, RaziUr Rehman, Israr Ahmed, "Pakistan Sign Language Recognition Using Statistical Template Matching", World Academy of Science, Engineering and Technology 3 2005.

[9]Philippe Dreuw, Pascal Steingrube, Thomas Deselaers, Hermann Ney. Smoothed Disparity Maps for Continuous American Sign Language Recognition. In: Pattern Recognition and Image Analysis LECT NOTES COMPUT SC 2009; 5524: 24-31.

[10]Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti. American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI '11, 2011; New York, NY, USA. pp. 279–286.

[11]Jayashree R. Pansare, Shravan H. Gawande, Maya Ingle. Real-Time Static Hand Gesture Recognition for American Sign Language (ASL) in Complex Background. In: Journal of Signal and Information Processing 2012; 3: 364-367 .

[12]Hervé Lahamy and Derek D. Lichti. Towards Real-Time and Rotation-Invariant American Sign Language Alphabet Recognition Using a Range Camera. In: Sensors 2012; 12: 14416-14441.

[13]Yun Li, Xiang Chen, Xu Zhang, Kongqiao Wang, and Z. Jane Wang.. A Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data. In: IEEE T BIO-MED ENG 2012; 59.

[14]Xiangwu Zhang, Dehui Kong, Lichun Wang, Jinghua Li, Yanfeng Sun, Qingming Huang. Synthesis of Chinese Sign Language Prosody Based on Head. In: International Conference on Computer Science & Service System (CSSS); 11-13 Aug. 2012; Nanjing: pp.1860 – 1864.

[15]Xing Guo, Zhenyu Lu, Rongbin Xu, Zhengyi Liu , Jianguo Wu. Big-screen text input system based on gesture recognition. In: Proceedings of the 2nd International Conference On Systems Engineering and Modeling (ICSEM-13), 2013; Paris; France.

[16]A. Samir Elons , Magdy Abull-ela, M.F. Tolba . A proposed PCNN features quality optimization technique for pose-invariant 3D Arabic sign language recognition. In: APPL SOFT COMPUT April 2013; 13: 1646–1660.

[17]Ershaed, I. Al-Alali, N. Khasawneh, and M. Fraiwan. An arabic sign language computer interface using the xbox kinect. In: Annual Undergraduate Research Conf. on Applied Computing May 4-5, 2011; Dubai, UAE.

[18]M. Mohandes, M. Deriche, U. Johar, S. Ilyas. A signer-independent Arabic Sign Language recognition system using face detection, geometric features, and a Hidden Markov Model. In: COMPUT ELECTR ENG March 2012; 38: 422–433.

[19]Tolba, M. F, Samir, Ahmed, Abul-Ela, Magdy. 3D Arabic sign language recognition using linear combination of multiple 2D views. In: 8th International Conference on Informatics and Systems (INFOS); 14-16 May 2012; Cairo: pp. MM-6 - MM-13.

[20]Jalilian, Bahare, and Abdolah Chalechale. "Persian Sign Language Recognition Using Radial Distance and Fourier Transform." International Journal of Image, Graphics and Signal Processing (IJIGSP) 6, no. 1 (2013): 40.

[21]Singh, Amarjot, John Buonassisi, and Sukriti Jain. "Autonomous Multiple Gesture Recognition System for Disabled People." International Journal of Image, Graphics and Signal Processing (IJIGSP) 6, no. 2 (2014): 39.

[22]A. Samir Elons , Magdy Abull-ela, M.F. Tolba , "A proposed PCNN features quality optimization technique for pose-invariant 3D Arabic sign language recognition", Applied Soft Computing, Volume 13, Issue 4, April 2013, Pages 1646–1660.

[23]S. Padam Priyal, Prabin Kumar Bora , "A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments" , Pattern Recognition, Volume 46, Issue 8, August 2013, Pages 2202–2219

[24]Yun Li, Xiang Chen, Xu Zhang, Kongqiao Wang, and Z. Jane Wang.," A Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data"Ieee Transactions On Biomedical Engineering, Vol. 59, No. 10, October 2012.

[25]Philippe Dreuw, Pascal Steingrube, Thomas Deselaers, Hermann Ney, "Smoothed Disparity Maps for Continuous American Sign Language Recognition" Pattern Recognition and Image Analysis Lecture Notes in Computer Science Volume 5524, 2009, pp 24-31

[26]Rini Akmeliawati, Melanie Po-Leen Ooi and Ye Chow Kuang, "Real-Time Malaysian Sign Language Translation using Colour Segmentation and Neural Network", IMTC 2007 - Instrumentation and Measurement Technology Conference Warsaw, Poland, 1-3, May 2007.

[27]Sumaira Kausar, M. Younus,Javed, Shaleeza,Sohail: "Recognition of gestures in Pakistani sign language using fuzzy classifier" , 8th International conference on Signal processing, computational geometry and artificial vision 2008 (ISCGAV'08), 20-22 August 2008, Rhodes, Greece, Pages 101-105

[28]Ershaed, I. Al-Alali, N. Khasawneh, and M. Fraiwan. An arabic sign language computer interface using the xbox kinect. In Annual Undergraduate Research Conf. on Applied Computing, May 2011.

[29]Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti. American sign language recognition with the kinect. In Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI '11, pages 279–286, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0641-6. doi: 10.1145/2070481.2070532.

[30]Helen Cooper, Eng-Jon Ong, Nicolas Pugeault, Richard Bowden, "Sign Language Recognition using Sub-Units", Journal of Machine Learning Research 13 (2012) 2205-2231.

[31]Xu shijian, Xu jie, Tang yijun, Shen likai, "Design of the Portable Gloves for Sign Language Recognition", Advances in Computer Science and Education Applications Communications in Computer and Information Science Volume 202, 2011, pp 171-177

[32]Tan Tian Swee, A.K. Ariff, S.-H. Salleh, Siew Kean Seng, Leong Seng Huat "Wireless data gloves Malay sign language recognition system", In proceeding of 6th International Conference on Information, Communications & Signal Processing, 2007;DOI: 10.1109/ICICS.2007.4449599 ISBN: 978-1-4244-0983-9

[33]Jayashree R. Pansare, Shravan H. Gawande, Maya Ingle, " Real-Time Static Hand Gesture Recognition for American Sign Language (ASL) in Complex Background", Journal of Signal and Information Processing, 2012, 3, 364-367 doi:10.4236/jsip.2012.33047 Published Online August 2012.

[34]Hervé Lahamy and Derek D. Lichti, "Towards Real-Time and Rotation-Invariant American Sign Language Alphabet Recognition Using a Range Camera", Sensors 2012, 12, 14416-14441; doi:10.3390/s121114416

[35]Xiangwu Zhang, Dehui Kong, Lichun Wang, Jinghua Li, Yanfeng Sun, Qingming Huang "Synthesis of Chinese Sign Language Prosody Based on Head" International Conference on Computer Science & Service System (CSSS), Nanjing, 2012, 11-13 Aug. 2012, pp:1860 – 1864, Print ISBN:978-1-4673-0721-5.

[36]Pakistan Association of Deaf, Pakistan, accessed at: http://www.padeaf.org, Accessed on: 20th June 2013.

[37]M-K. Hu.Visual pattern recognition by moment invariants. IRE Trans. on Information Theory, IT-8:pp. 179-187, 1962.

International Journal of Image, Graphics and Signal Processing (IJIGSP)