International Journal of Intelligent Systems and Applications (IJISA)
IJISA Vol. 12, No. 6, 8 Dec. 2020
Cover page and Table of Contents: PDF (size: 651KB)
Deep Learning Sign Language Recognition System Based on Wi-Fi CSI
Full Text (PDF, 651KB), PP.33-45
Views: 0 Downloads: 0
Author(s)
Marwa R. M. Bastwesy
1,*
Nada M. ElShennawy
1
Mohamed T. Faheem Saidahmed
1
Index Terms
Wireless, Device-free sensing, Channel State Information, Sign Language Recognition, Deep Learning, WiFi Imaging
Abstract
Many sensing gesture recognition systems based on Wi-Fi signals are introduced because of the commercial off-the-shelf Wi-Fi devices without any need for additional equipment. In this paper, a deep learning-based sign language recognition system is proposed. Wi-Fi CSI amplitude and phase information is used as input to the proposed model. The proposed model uses
three types of deep learning: CNN, LSTM, and ABLSTM with a complete study of the impact of optimizers, the use of amplitude and phase of CSI, and preprocessing phase. Accuracy, F-score, Precision, and recall are used as performance metrics to evaluate the proposed model. The proposed model achieves 99.855%, 99.674%, 99.734%, and 93.84% average recognition accuracy for the lab, home, lab + home, and 5 different users in a lab environment, respectively. Experimental results show that the proposed model can effectively detect sign gestures in complex environments compared with some deep learning recognition models.
Cite This Paper
Marwa R. M. Bastwesy, Nada M. ElShennawy, Mohamed T. Faheem Saidahmed, "Deep Learning Sign Language Recognition System Based on Wi-Fi CSI", International Journal of Intelligent Systems and Applications(IJISA), Vol.12, No.6, pp.33-45, 2020. DOI:10.5815/ijisa.2020.06.03
Reference
[1]H. Abdelnasser, M. Youssef, and K. A. Harras,“Wigest A ubiquitous wifi-based gesture recognition system,” in 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, 2015, pp. 1472–1480.
[2]Y. Wang, K. Wu, and L. M. Ni, “Wifall: Device-free fall detection by wireless networks,” IEEE Transactions on Mobile Computing, vol. 16, no. 2, pp. 581–594, 2016.
[3]T. Z. Chowdhury, “Using wi-fi channel state information (csi) for human activity recognition and fall detection,” Ph.D. dissertation, University of British Columbia, 2018.
[4]S. Oprisescu, C. Rasche, and B. Su, “Automatic static hand gesture recognition using tof cameras,” in 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). IEEE, 2012, pp.2748–2751.
[5]Z. Zhang, “Microsoft kinect sensor and its effect,” IEEE multimedia,vol. 19, no. 2, pp. 4–10, 201
[6]U. of Washington. (2016) Signaloud demo. [Online]. Available: http://www.washington.edu/news/2016/04/12/uw-undergraduate-teamwins-10000-lemelson-mit-student-prize-for-gloves-that-translate-signlanguage/
[7]M. Scholz, S. Sigg, H. R. Schmidtke, and M. Beigl, “Challenges for device-free radio-based activity recognition,” in Proceedings of the 3rd workshop on Context Systems, Design, Evaluation and Optimisation (CoSDEO 2011), in Conjunction with MobiQuitous, vol. 2011, 2011.
[8]M. Al-qaness and F. Li, “Wiger: Wifi-based gesture recognition system,” ISPRS International Journal of Geo-Information, vol. 5, no. 6, p. 92, 2016.
[9]Q. Pu, S. Gupta, S. Gollakota, and S. Patel, “Whole-home gesture recognition using wireless signals,” in Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 2013, pp. 27–38.
[10]Y. Ma, G. Zhou, S. Wang, H. Zhao, and W. Jung, “Signfi: Sign language recognition using wifi,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2, no. 1, p. 23, 2018.
[11]J. Wang, D. Vasisht, and D. Katabi, “Rf-idraw: virtual touch screen in the air using rf signals,” ACM SIGCOMM Computer Communication Review, vol. 44, no. 4, pp. 235–246, 2015.
[12]S. Sigg, S. Shi, F. Buesching, Y. Ji, and L. Wolf, “Leveraging rf-channel fluctuation for activity recognition: Active and passive systems, continuous and rssi-based signal features,” in Proceedings of International Conference on Advances in Mobile Computing & Multimedia, 2013, pp. 43–52.
[13]W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, “Understanding and modeling of wifi signal based human activity recognition,” in Proceedings of the 21st annual international conference on mobile computing and networking. ACM, 2015, pp. 65–76.
[14]W. He, K. Wu, Y. Zou, and Z. Ming, “Wig: Wifi-based gesture recognition system,” in 2015 24th International Conference on Computer Communication and Networks (ICCCN). IEEE, 2015, pp. 1–7.
[15]H. Li, W. Yang, J. Wang, Y. Xu, and L. Huang, “Wifinger: talk to your smart devices with finger-grained gesture,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2016, pp. 250–261.
[16]Z. Tian, J. Wang, X. Yang, and M. Zhou, “Wicatch: a wi-fi based hand gesture recognition system,” IEEE Access, vol. 6, pp. 16 911–16 923, 2018.
[17]C. Wang, S. Chen, Y. Yang, F. Hu, F. Liu, and J. Wu, “Literature review on wireless sensing-wi-fi signal-based recognition of human activities,” Tsinghua Science and Technology, vol. 23, no. 2, pp. 203–222, 2018.
[18]C. Feng, S. Arshad, and Y. Liu, “Mais: Multiple activity identification system using channel state information of wifi signals,” in International Conference on Wireless Algorithms, Systems, and Applications. Springer, 2017, pp. 419–432.
[19]Y. Xie, Z. Li, and M. Li, “Precise power delay profiling with commodity wi-fi,” IEEE Transactions on Mobile Computing, vol. 18, no. 6, pp. 1342–1355, 2018.
[20]S. Liu, Y. Zhao, F. Xue, B. Chen, and X. Chen, “Deepcount: Crowd counting with wifi via deep learning,” arXiv preprint arXiv:1903.05316, 2019.
[21]C. Wu, Z. Yang, Z. Zhou, K. Qian, Y. Liu, and M. Liu, “Phaseu: Realtime los identification with wifi,” in 2015 IEEE conference on computer communications (INFOCOM). IEEE, 2015, pp. 2038–2046.
[22]C. Gao, Y. Li, and X. Zhang, “Livetag: Sensing human object interaction through passive chipless wifi tags,” in 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18), 2018, pp. 533–546.
[23]L. Bottou, “Stochastic gradient descent tricks,” in Neural networks: Tricks of the trade. Springer, 2012, pp. 421–436.
[24]B. Wei, K. Li, C. Luo, W. Xu, and J. Zhang, “No need of data preprocessing: A general framework for radio-based device-free context awareness,” arXiv preprint arXiv:1908.03398, 2019.
[25]S. Yousefi, H. Narui, S. Dayal, S. Ermon, and S. Valaee, “A survey on behavior recognition using wifi channel state information,” IEEE Communications Magazine, vol. 55, no. 10, pp. 98–104, 2017.
[26]Z. Chen, L. Zhang, C. Jiang, Z. Cao, and W. Cui, “Wifi csi based passive human activity recognition using attention based blstm,” IEEE Transactions on Mobile Computing, 2018.
[27]A. Gulli and S. Pal, Deep learning with Keras. Packt Publishing Ltd, 2017.
[28]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
[29]M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701, 2012.