International Journal of Intelligent Systems and Applications (IJISA)
IJISA Vol. 14, No. 6, 8 Dec. 2022
Cover page and Table of Contents: PDF (size: 658KB)
Covid-19 Control: Face Mask Detection Using Deep Learning for Balanced and Unbalanced Dataset
Full Text (PDF, 658KB), PP.50-62
Views: 0 Downloads: 0
Author(s)
Index Terms
Covid-19, Facemask, FPS, Haar-cascade, IoU, mAP, Mask-Wearing, MTCNN, YOLOv4
Abstract
Facemask wearing is becoming a norm in our daily lives to curb the spread of Covid-19. Ensuring facemasks are worn correctly is a topic of concern worldwide. It could go beyond manual human control and enforcement, leading to the spread of this deadly virus and many cases globally. The main aim of wearing a facemask is to curtail the spread of the covid-19 virus, but the biggest concern of most deep learning research is about who is wearing the mask or not, and not who is incorrectly wearing the facemask while the main objective of mask wearing is to prevent the spread of the covid-19 virus. This paper compares three state-of-the- art object detection approaches: Haarcascade, Multi-task Cascaded Convolutional Networks (MTCNN), and You Only Look Once version 4 (YOLOv4) to classify who is wearing a mask, who is not wearing a mask, and most importantly, who is incorrectly wearing the mask in a real-time video stream using FPS as a benchmark to select the best model. Yolov4 got about 40 Frame Per Seconds (FPS), outperforming Haarcascade with 16 and MTCNN with 1.4. YOLOv4 was later used to compare the two datasets using Intersection over Union (IoU) and mean Average Precision (mAP) as a comparative measure; dataset2 (balanced dataset) performed better than dataset1 (unbalanced dataset). Yolov4 model on dataset2 mapped and detected images of masks worn incorrectly with one correct class label rather than giving them two label classes with uncertainty in dataset1, this work shows the advantage of having a balanced dataset for accuracy. This work would help decrease human interference in enforcing the COVID-19 face mask rules and create awareness for people who do not comply with the facemask policy of wearing it correctly. Hence, significantly reducing the spread of COVID-19.
Cite This Paper
Ademola A. Adesokan, "Covid-19 Control: Face Mask Detection Using Deep Learning for Balanced and Unbalanced Dataset", International Journal of Intelligent Systems and Applications(IJISA), Vol.14, No.6, pp.50-62, 2022. DOI:10.5815/ijisa.2022.06.05
Reference
[1]World Health Organization 2021. Coronavirus disease (COVID-19) – World Health Organization. [online] Available at: who.int [Accessed 14 December 2021].
[2]H. Adusumalli, D. Kalyani, R. K. Sri, M. Pratapteja and P. V. R. D. P. Rao, “Face Mask Detection Using OpenCV,” 2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), 2021, pp. 1304-1309, doi: 10.1109/ICICV50876.2021.9388375.
[3]“Weekly epidemiological update on COVID-19 - 14 December 2021,” Who.int. [Online]. Available: https://www.who.int/publications/m/item/ weekly-epidemiological-update-on-covid-19 14-december-2021. [Accessed: 14-Dec-2021].
[4]“COVID-19 vaccines: Myth versus fact,” Hop- kinsmedicine.org.[Online]. Available: https://www.hopkinsmedicine.org/health/conditions-and-diseases/coronavirus/covid-19-vaccines-myth-versus-fact. [Accessed: 14-Dec-2021].
[5]CDC, “Possible side effects after getting a COVID-19 vaccine,” Centers for Disease Control and Prevention, 14-Dec-2021. [Online]. Available:https://www.cdc.gov/coronavirus/2019-ncov/vaccines/expect/ after.html. [Accessed: 14-Dec-2021].
[6]H. Ritchie et al., “Coronavirus Pandemic (COVID-19),” Our World in Data, 2020.
[7]A. Negi, P. Chauhan, K. Kumar and R. S. Rajput, “Face Mask Detection Classifier and Model Pruning with Keras-Surgeon,” 2020 5th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE), 2020, pp. 1-6, doi: 10.1109/ICRAIE51050.2020.9358337.
[8]M. S. Ejaz, M. R. Islam, M. Sifatullah and A. Sarker, “Implementation of Principal Component Analysis on Masked and Non-masked Face Recognition”, 2019 1st International Conference on Advances in Science Engineering and Robotics Technology (ICASERT), pp. 1-5, 2019.
[9]BOSHENG QIN and DONGXIAO LI, “Identifying Facemask- wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID- 19”, May 2020, [online] Available: https://doi.org/10.21203/rs.3.rs-28668/v1+].
[10]“Face Mask Detection,” kaggle.com. https://www.kaggle.com/ andrewmvd/face-mask-detection.
[11]Adnane Cabani, Karim Hammoudi, Halim Benhabiles, and Mahmoud Melkemi, “MaskedFace-Net — A dataset of correctly/incorrectly masked face images in the context of COVID-19” Smart Health, ISSN 2352–6483, Elsevier, 2020, DOI:10.1016/j.smhl.2020.100144
[12]Karim Hammoudi, Adnane Cabani, Halim Benhabiles, and Mahmoud Melkemi,”Validating the correct wearing of protection mask by taking a selfie: design of a mobile application “CheckYourMask” to limit the spread of COVID-19”, CMES-Computer Modeling in Engineering Sciences, Vol.124, №3, pp. 1049–1059, 2020, DOI:10.32604/cmes.2020.011663.
[13]O. Cakiroglu, C. Ozer, and B. Gunsel, “Design of a deep face detector by mask r-cnn,” in 2019 27th Signal Processing and Communications Applications Conference (SIU). IEEE, 2019, pp. 1–4.
[14]T. Meenpal, A. Balakrishnan, and A. Verma, “Facial mask detection using semantic segmentation,” in 2019 4th International Conference on Computing, Communications and Security (ICCCS). IEEE, 2019, pp. 1–5.
[15]M. R. Bhuiyan, S. A. Khushbu, and M. S. Islam, “A deep learning based assistive system to classify covid-19 face mask for human safety with yolov3,” in 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT). IEEE, 2020, pp. 1–5.
[16]A. S. Joshi, S. S. Joshi, G. Kanahasabai, R. Kapil, and S. Gupta, “Deep learning framework to detect face masks from video footage,” in 2020 12th International Conference on Computational Intelligence and Communication Networks (CICN). IEEE, 2020, pp. 435–440.
[17]Y. Wang, B. Luo, J. Shen, and M. Pantic, “Face mask extraction in video sequence,” International Journal of Computer Vision, vol. 127, no. 6-7, pp. 625–641, 2019.
[18]A. Chavda, J. Dsouza, S. Badgujar, and A. Damani, “Multi-stage cnn architecture for face mask detection,” arXiv preprint arXiv:2009.07627.
[19]S. Abbasi, H. Abdi and A. Ahmadi, “A Face-Mask Detection Approach based on YOLO Applied for a New Collected Dataset,” 2021 26th International Computer Conference, Computer Society of Iran (CSICC), 2021, pp. 1-6, doi: 10.1109/CSICC52343.2021.9420599.
[20]S. Susanto, F. A. Putra, R. Analia and I. K. L. N. Suciningtyas, “The Face Mask Detection for Preventing the Spread of COVID-19 at Politeknik Negeri Batam,” 2020 3rd International Conference on Applied Engineering (ICAE), 2020, pp. 1-5, doi: 10.1109/ICAE50557.2020.9350556.
[21]W. Vijitkunsawat and P. Chantngarm, “Study of the Performance of Machine Learning Algorithms for Face Mask Detection,” 2020 - 5th International Conference on Information Technology (InCIT), 2020, pp. 39-43, doi: 10.1109/InCIT50588.2020.9310963.
[22]G. Deore, R. Bodhula, V. Udpikar and V. More, “Study of masked face detection approach in video analytics,” 2016 Conference on Advances in Signal Processing (CASP), 2016, pp. 196-200, doi: 10.1109/CASP.2016.7746164.
[23]X. Peng, H. Zhuang, G. -B. Huang, H. Li and Z. Lin, “Robust Real-time Face Tracking for People Wearing Face Masks,” 2020 16th International Conference on Control, Automation, Robotics and Vision (ICARCV), 2020, pp. 779-783, doi: 10.1109/ICARCV50220.2020.9305356.
[24]Y. Meng, N. Liu, Z. Su, X. Wang and H. Wang, “RESEARCH ON REAL-TIME DETECTION METHOD OF FACE WEARING MASK WITH LARGE TRAFFIC BASED ON DEEP LEARNING,” The 8th International Symposium on Test Automation Instrumentation (ISTAI 2020), 2020, pp. 121-126, doi: 10.1049/icp.2021.1338.
[25]S. Asif, Y. Wenhui, Y. Tao, S. Jinhai and K. Amjad, “Real Time Face Mask Detection System using Transfer Learning with Machine Learning Method in the Era of Covid-19 Pandemic,” 2021 4th International Conference on Artificial Intelligence and Big Data (ICAIBD), 2021,
[26]M. S. Ejaz and M. R. Islam, “Masked Face Recognition Using Convolutional Neural Network,” 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), 2019, pp. 1-6, doi: 10.1109/STI47673.2019.9068044.
[27]P. Hofer, M. Roland, P. Schwarz, M. Schwaighofer and R. Mayrhofer, “Importance of different facial parts for face detection networks,” 2021 IEEE International Workshop on Biometrics and Forensics (IWBF), 2021, pp. 1-6, doi: 10.1109/IWBF50991.2021.9465087.
[28]R. K. Kodali and R. Dhanekula, “Face Mask Detection Using Deep Learning,” 2021 International Conference on Computer Communication and Informatics (ICCCI), 2021, pp. 1-5, doi: 10.1109/ICCCI50826.2021.9402670.
[29]W. Jian and L. Lang, “Face mask detection based on Transfer learning and PP-YOLO,” 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), 2021, pp. 106-109, doi: 10.1109/ICBAIE52039.2021.9389953.
[30]J. Gathani and K. Shah, “Detecting Masked Faces using Region- based Convolutional Neural Network,” 2020 IEEE 15th International Conference on Industrial and Information Systems (ICIIS), 2020, pp. 156-161, doi: 10.1109/ICIIS51140.2020.9342737.
[31]J. Negi, K. Kumar, P. Chauhan and R. S. Rajput, “Deep Neural Architecture for Face mask Detection on Simulated Masked Face Dataset against Covid-19 Pandemic,” 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), 2021, pp. 595-600, doi: 10.1109/ICCCIS51004.2021.9397196.
[32]R. Liu and Z. Ren, “Application of Yolo on Mask Detection Task,” 2021 IEEE 13th International Conference on Computer Research and Development (ICCRD), 2021, pp. 130-136, doi: 10.1109/ICCRD51685.2021.9386366.
[33]P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001, pp. I-I, doi: 10.1109/CVPR.2001.990517.
[34]“OpenCV: Cascade Classifier,” Opencv.org. [Online]. Available: https://docs.opencv.org/3.4/db/d28/tutorial cascade classifier.html. [Accessed: 14-Dec-2021].
[35]R. Gradilla, “Multi-task cascaded convolutional networks (MTCNN) for face detection and facial landmark alignment,” Medium, 27-Jul-2020. [Online]. Available: MTCNN on medium. [Accessed: 14-Dec-2021].
[36]K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multi-task cascaded convolutional networks,” arXiv [cs.CV], 2016.
[37]C. Supeshala, “YOLO v4 or YOLO v5 or PP-YOLO? Which should I use?” Towards Data Science, 23-Aug-2020. [Online]. Available: https:// towardsdatascience.com/yolo-v4-or-yolo-v5-or-pp-yolo-dad8e40f7109. [Accessed: 14-Dec-2021].
[38]J. Solawetz, “Breaking down YOLOv4,” Roboflow Blog, 04-Jun-2020. [Online]. Available: https://blog.roboflow.com/ a-thorough-breakdown-of-yolov4/. [Accessed: 14-Dec-2021].
[39]Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[40]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
[41]Xianzhi Du, Tsung-Yi Lin, Pengchong Jin, Golnaz Ghiasi, Mingxing Tan, Yin Cui, Quoc V Le, and Xiaodan Song. SpineNet: Learning scale-permuted backbone for recognition and localization. arXiv preprint arXiv:1912.05027, 2019.
[42]Mingxing Tan and Quoc V Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of International Conference on Machine Learning (ICML), 2019.
[43]Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh. CSPNet: A new backbone that can enhance learning capability of cnn. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop (CVPR Workshop), 2020.
[44]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 37(9):1904–1916, 2015.
[45]Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40(4):834–848, 2017.
[46]Songtao Liu, Di Huang, et al. Receptive field block net for accurate and fast object detection. In Proceedings of the European Conference on Computer Vision (ECCV), pages 385–400, 2018.
[47]Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), pages 3–19, 2018.
[48]Tsung-Yi Lin, Piotr Doll´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2117–2125, 2017.
[49]Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia. Path aggregation network for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8759–8768, 2018.
[50]Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. NAS-FPN: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7036–7045, 2019.
[51]Mingxing Tan, Ruoming Pang, and Quoc V Le. Efficient-Det: Scalable and efficient object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[52]Songtao Liu, Di Huang, and YunhongWang. Learning spatial fusion for single-shot object detection. arXiv preprint arXiv:1911.09516, 2019.
[53]Qijie Zhao, Tao Sheng, Yongtao Wang, Zhi Tang, Ying Chen, Ling Cai, and Haibin Ling. M2det: A single-shot object detector based on multi-level feature pyramid network. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 33, pages 9259–9266, 2019.
[54]Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems (NIPS), pages 91–99, 2015.
[55]Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. SSD: Single shot multibox detector. In Proceedings of the European Conference on Computer Vision (ECCV), pages 21–37, 2016.
[56]Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779–788, 2016.
[57]Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980–2988, 2017.
[58]Hei Law and Jia Deng. CornerNet: Detecting objects as paired keypoints. In Proceedings of the European Conference on Computer Vision (ECCV), pages 734–750, 2018.
[59]Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. CenterNet: Keypoint triplets for object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 6569–6578, 2019.
[60]Abdullah Rashwan, Agastya Kalra, and Pascal Poupart. Matrix Nets: A new deep architecture for object detection. In Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCV Workshop), pages 0–0, 2019.
[61]Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: Fully convolutional one-stage object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 9627–9636, 2019.
[62]Krishna Kumar Singh, Hao Yu, Aron Sarmasi, Gautam Pradeep, and Yong Jae Lee. Hide-and-Seek: A data augmentation technique for weakly-supervised localization and beyond. arXiv preprint arXiv:1811.02545, 2018.
[63]Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-FCN: Object detection via region-based fully convolutional networks. In Advances in Neural Information Processing Systems
[64]Kaiming He, Georgia Gkioxari, Piotr Doll´ar, and Ross Girshick. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2961–2969, 2017.
[65]Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. RepPoints: Point set representation for object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 9657–9666, 2019
[66]Cartucho, “OpenLabeling/README.md at master · Cartu- cho/openlabeling,” GitHub. [Online]. Available:https://github.com/Cartucho/OpenLabeling/blob/master/README.md. [Accessed: 14-Dec- 2021].
[67]T. Pariwat and P. Seresangtakul, “Multi-stroke Thai finger-spelling Sign language recognition system with deep learning,” MDPI, 04-Feb-2021. Available: https://www.mdpi.com/2073-8994/13/2/262/htm. [Accessed: 14-Dec-2021].