International Journal of Image, Graphics and Signal Processing (IJIGSP)
IJIGSP Vol. 16, No. 4, 8 Aug. 2024
Cover page and Table of Contents: PDF (size: 1351KB)
Fetal Brain Planes Classification Using Deep Ensemble Transfer Learning from U-Net Segmented Fetal Neurosonography Images
PDF (1351KB), PP.74-86
Views: 0 Downloads: 0
Author(s)
Index Terms
Fetal Brain Planes Classification, U-Net Based Segmentation, Deep Ensemble Transfer Learning, Grad-CAM
Abstract
Fetal neurosonography is potentially used to examine the fetal brain by scanning the trans-thalamic (TT), trans-cerebellum (TC), and trans-ventricular (TV) planes. Cross-sectional analysis of these planes is useful to assess the brain anatomy, development, and abnormality for intervention and treatment plans even at the postnatal stage. To minimize the errors and processing time involved in the traditional manual subjective approach, the automatic classification of fetal brain planes is crucial. In this study, a deep learning-based method for automatically categorizing fetal brain planes from ultrasound images is proposed and evaluated. Firstly, the brain region has been segmented from the fetal brain ultrasound images using U-Net to prepare an efficient data set for the classifier model. Then, an ensemble convolutional neural network (CNN) model including well-known Inception V3, ResNet50-V2, and DenseNet-201 models with max voting is designed to classify the segmented brain planes. 2019 fetal brain ultrasound images from a widely used publicly accessible experts-annotated dataset are used to evaluate the performance of the proposed framework. The obtained results analysis shows that using the segmented images as input improves the performance of the classifier from its raw form. The gradient class activation mapping (Grad-CAM) based inspection shows noteworthy localization capability of the last convolution layer. The ensemble model has also outperformed its individual model’s performance. The suggested categorization framework is satisfactory compared to related recent works, with a testing accuracy of 97.68%. The proposed framework for fetal brain plane classification is expected to be useful for clinical applications.
Cite This Paper
Md. Nazmul Hasan, A. B. M. Aowlad Hossain, "Fetal Brain Planes Classification Using Deep Ensemble Transfer Learning from U-Net Segmented Fetal Neurosonography Images", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.16, No.4, pp. 74-86, 2024. DOI:10.5815/ijigsp.2024.04.06
Reference
[1]L. M. Carlson and L. V. Neeta, “Prenatal diagnosis: screening and diagnostic tool,” Obstetrics and Gynecology Clinics,vol. 44, no. 2, pp. 245-256, 2017. doi: 10.1016/j.ogc.2017.02.004.
[2]H. J. F. Milani, et al, “Ultrasonographic evaluation of the fetal central nervous system: a review of guidelines,” Radiol Bras, vol 52, on. 3, pp 176–181, 2019. doi: 10.1590/0100-3984.2018.0056.
[3]T. L. A. Van Den Heuvel, H. Petros, S. A. Santini, C. L. De Korte, and B. Van Ginneken, “Automated Fetal Head Detection and Circumference Estimation from Free-Hand Ultrasound Sweeps Using Deep Learning in Resource-Limited Countries,” Ultrasound in Medicine and Biology, vol. 45, no. 3, pp. 773–785, Mar. 2019, doi: 10.1016/j.ultrasmedbio.2018.09.015.
[4]M. Whitworth, L. Bricker, and C. Mullan, “Ultrasound for fetal assessment in early pregnancy,” The Cochrane Library, vol. 2015, no. 7, Jul. 2015, doi: 10.1002/14651858.cd007058.pub3.
[5]T. Dias, S. Sairam, and S. Kumarasiri, “Ultrasound diagnosis of fetal renal abnormalities,” Best Practice & Research Clinical Obstetrics & Gynaecology, vol. 28, no. 3, pp. 403–415, 2014. doi: 10.1016/j.bpobgyn.2014.01.009
[6]H. H. Bijma, A. van der Heide, and H. I. Wildschut, “Decision-making after ultrasound diagnosis of fetal abnormality,” Reproductive Health Matters, vol. 16, no. 31, pp. 82–89, 2008. doi: 10.1007/s11296-007-0070-0
[7]D Abed M. Sonography 2nd Trimester Assessment, Protocols, and Interpretation. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; Jan 2023
[8]J. Yanase and E. Triantaphyllou, “A systematic survey of computer-aided diagnosis in medicine: Past and present developments,” Expert Systems with Applications, vol. 138, p. 112821, Dec. 2019, doi: 10.1016/j.eswa.2019.112821.
[9]P. Sridar et al, “Automatic identification of multiple planes of a fetal organ from 2-D ultrasound images,” In Proc. MICCAI Workshop Perinatal, Preterm Paediatric Image Anal. (PIPPI), pp. 1-10, 2016.
[10]H.-N. Xie et al., “Using deep‐learning algorithms to classify fetal brain ultrasound images as normal or abnormal,” Ultrasound in Obstetrics & Gynecology, vol. 56, no. 4, pp. 579–587, Oct. 2020, doi: 10.1002/uog.21967.
[11]M. Alzubaidi, M. Agus, U. Shah, M. Makhlouf, K. Alyafei, and M. Househ, “Ensemble Transfer Learning for Fetal Head Analysis: From segmentation to gestational age and weight prediction,” Diagnostics, vol. 12, no. 9, p. 2229, Sep. 2022, doi: 10.3390/diagnostics12092229.
[12]J. Li et al., “Automatic fetal head circumference measurement in ultrasound using random forest and fast ellipse fitting,” IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 1, pp. 215–223, Jan. 2018, doi: 10.1109/jbhi.2017.2703890.
[13]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, May 2017, doi: 10.1145/3065386.
[14]G. Litjens et al., “A survey on deep learning in medical image analysis,” Medical Image Analysis, vol. 42, pp. 60–88, Dec. 2017, doi: 10.1016/j.media.2017.07.005.
[15]A. B. M. Aowlad Hossain, Jannatul Kamrun Nisha, Fatematuj Johora, “Breast Cancer Classification from Ultrasound Images using VGG16 Model based Transfer Learning,” International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.15, No.1, pp. 12-22, 2023. doi:10.5815/ijigsp.2023.01.02
[16]S. Liu et al., “Deep Learning in Medical Ultrasound Analysis: A review,” Engineering, vol. 5, no. 2, pp. 261–275, Apr. 2019, doi: 10.1016/j.eng.2018.11.020.
[17]O. Ronneberger, P. Fischer, and T. Brox, “U-NET: Convolutional Networks for Biomedical Image Segmentation,” in Lecture Notes in Computer Science, 2015, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28.
[18]X. Yin, L. Sun, Y. Fu, R. Lu, and Y. Zhang, “U-Net-Based medical image segmentation,” Journal of Healthcare Engineering, vol. 2022, pp. 1–16, Apr. 2022, doi: 10.1155/2022/4189781
[19]S. S. Islam and A. B. M. A. Hossain, “Glioma Segmentation from Multimodal MRI using U-Net Convolutional Neural Networks,” in 2021 IEEE 2nd International Conference on Applied Electromagnetics, Signal Processing, & Communication (AESPC), Bhubaneswar, India, 2021, pp. 1-5, doi: 10.1109/AESPC52704.2021.9708467.
[20]V. B. Shereena, and G. Raju, “Medical Ultrasound Image Segmentation Using U-Net Architecture” In Singh, M., Tyagi, V., Gupta, P.K., Flusser, J., Ören, T. (eds) Advances in Computing and Data Sciences. ICACDS 2022. Communications in Computer and Information Science, vol 1613. Springer, Cham, 2022. doi:10.1007/978-3-031-12638-3_30
[21]X. P. Burgos-Artizzu et al., “Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes,” Scientific Reports, vol. 10, no. 1, Jun. 2020, doi: 10.1038/s41598-020-67076-5.
[22]N. Rahmayanti, H. Pradani, M. Pahlawan, and R. Vinarti, “Comparison of machine learning algorithms to classify fetal health using cardiotocogram data,” in Sixth Information Systems International Conference (ISICO 2021), pp. 162–171, 2022.
[23]B. Zhang, H. Liu, H. Luo, and K. Li, “Automatic quality assessment for 2D fetal sonographic standard plane based on multitask learning,” Medicine, vol. 100, no. 4, pp. e24427, Jan. 2021, doi: 10.1097/md.0000000000024427.
[24]D. Coronado-Gutierrez et al. “Automatic classification of fetal axial brain planes from ultrasound images via deep learning,” ISUOG 2020.
[25]M. A. Ganaie, M. Hu, M. Tanveer, and P. N. Suganthan, “Ensemble deep learning: A review,” Engineering Applications of Artificial Intelligence, vol. 115, pp. 105151, Oct. 2022, doi: 10.1016/j.engappai.2022.105151.
[26]M. Foysal and A. B. M. Aowlad Hossain, “COVID-19 Detection from Chest CT Images using Ensemble Deep Convolutional Neural Network,” 2021 2nd International Conference for Emerging Technology (INCET), Belagavi, India, 2021, pp. 1-6, doi: 10.1109/INCET51464.2021.9456387.
[27]M. Alzubaidi, M. Agus, U. Shah, M. Makhlouf, K. Alyafei, and M. Househ, “Ensemble Transfer Learning for Fetal Head Analysis: From segmentation to gestational age and weight prediction,” Diagnostics, vol. 12, no. 9, pp. 2229, Sep. 2022, doi: 10.3390/diagnostics12092229.
[28]D. R. Sarvamangala and R. V. Kulkarni, “Convolutional neural networks in medical image understanding: a survey,” Evol. Intel., vol. 15, pp. 1–22, 2022. doi: 10.1007/s12065-020-00540-3.
[29]S. S. Yadav and S. M. Jadhav, “Deep convolutional neural network based medical image classification for disease diagnosis,” Journal of Big Data, vol. 6, no. 1, Dec. 2019, doi: 10.1186/s40537-019-0276-2.
[30]Y. LeCun, Y. Bengio, and G. E. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015, doi: 10.1038/nature14539.
[31]C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on Deep transfer learning,” in Lecture Notes in Computer Science, 2018, pp. 270–279. doi: 10.1007/978-3-030-01424-7_27
[32]H. Ravishankar et al., “Understanding the mechanisms of deep transfer learning for medical images,” arXiv (Cornell University), Apr. 2017, [Online]. Available: https://arxiv.org/pdf/1704.06040
[33]C. Szegedy, W. Liu, Y. Jia, P. Sermanet, and S. Reed, “Going Deeper with Convolutions,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, 2015, doi: 10.1109/CVPR.2015.7298594.
[34]K. He, X. Zhang, S. Ren, J. Sun, “Identity mappings in deep residual networks,” in European conference on computer vision, Springer, 2016, pp. 630–645.
[35]G. H. Z. L. L. Van Der Maaten and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2261–2269, doi: 10.1109/CVPR.2017.243.
[36]A Tharwat, “Classification assessment methods,” Appl Comput Inform vol. 17, pp. 168, 2018.
[37]J. Ye, B. Zou, H. Zhang, n. zhan, C. Han, X. Chenet al., “A deep convolutional neural network based hybrid framework for fetal head standard plane identification,”2020, doi: 10.22541/au.158879105.54189382.
[38]M. Foysal, A. B. M. A. Hossain, A Yassine, M. S. Hossain, “Detection of COVID-19 Case from Chest CT Images Using Deformable Deep Convolutional Neural Network,” vol. 2023, Journal of Healthcare Engineering, 2023, doi: 10.1155/2023/4301745.