International Journal of Image, Graphics and Signal Processing (IJIGSP)
IJIGSP Vol. 16, No. 4, 8 Aug. 2024
Cover page and Table of Contents: PDF (size: 579KB)
Autism Spectrum Disorder Screening on Home Videos Using Deep Learning
PDF (579KB), PP.106-115
Views: 0 Downloads: 0
Author(s)
Index Terms
Autism Spectrum Disorder, Deep Learning, CNN, LSTM
Abstract
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by difficulty in social interactions, communication, and repetitive behaviors. Protocols like ADOS (Autism Diagnostic Observation Schedule) and ADI-R (Autism Diagnostic Interview Revised) are used by experts to assess the subject’s behavior which is time-consuming. Over the decade, researchers have studied the application of various Machine Learning techniques for ASD diagnosis through facial feature analysis, eye movement tracking, questionnaire analysis, functional magnetic resonance imaging (fMRI) analysis, etc. However, these techniques are not helpful for the parent or guardian of the child to perform an initial screening. This research proposes a novel deep learning model to diagnose ASD using general videos of the subject performing some tasks with the parent/guardian. Since there is no publicly available dataset on ASD videos, a dataset is created by collecting the videos of autistic children performing some activities with parents/guardians from YouTube from different demographic locations. These videos are then converted to skeletal key points to extract the child's engagement and social interaction in a given task. The proposed CNN-LSTM model is trained on 80% of the collected videos and then tested on the remaining 20%. The experiment results on various combinations of pre-trained CNN models and LSTM/BiLSTM show that the proposed model can be used as an initial autism screening tool. Among the different combinations, the MobileNet and Bi-LSTM combo achieved the best test accuracy of 84.95% with 89% precision, recall and F1-score.
Cite This Paper
Anjali Singh, Abha Rawat, Mitali Laroia, Seeja K. R. , "Autism Spectrum Disorder Screening on Home Videos Using Deep Learning", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.16, No.4, pp. 106-115, 2024. DOI:10.5815/ijigsp.2024.04.08
Reference
[1]National Institute of Mental Health (NIMH), Autism Spectrum Disorder, https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd
[2]DSM Library, Diagnostic and Statistical Manual of Mental Disorders, https://www.psychiatry.org/psychiatrists/practice/dsm
[3]Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012). Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) manual (Part I): Modules 1-4. Western Psychological Services.
[4]Schopler, E., Reichler, R. J., & Renner, B. R. (1988). Childhood Autism Rating Scale (CARS). Western Psychological Services.
[5]Mahmoudi, M., Ahmadlou, M., Jahani, S., Aarabi, A., & Khaleghi, A. (2019). EEG-based classification of autism using dynamic functional connectivity: a deep learning approach. Journal of neural engineering, 16(5), 056026.
[6]Zhao, Z., Tang, H., Zhang, X., Qu, X., Hu, X., & Lu, J. (2021). Classification of children with autism and typical development using eye-tracking data from face-to-face conversations: Machine learning model development and performance evaluation. Journal of Medical Internet Research, 23(8), e29328. https://doi.org/10.2196/29328
[7]Guillon, Q., Hadjikhani, N., Baduel, S., & Rogé, B. (2014). Visual social attention in autism spectrum disorder: insights from eye tracking studies. Neuroscience & Biobehavioral Reviews, 42, 279-297.
[8]Xie, J., Wang, L., Webster, P., Yao, Y., Sun, J., Wang, S., & Zhou, H. (2019). A two-stream end-to-end deep learning network for recognizing atypical visual attention in autism spectrum disorder. arXiv preprint arXiv:1911.11393. https://doi.org/10.48550/arXiv.1911.11393
[9]Mit Saliency Benchmark, Springer International Publishing. http://saliency.mit.edu/datasets.html, Accessed 26 August 2022
[10]Wu, C., Liaqat, S., Cheung, S. C., Chuah, C. N., & Ozonoff, S. (2019, July). Predicting autism diagnosis using image with fixations and synthetic saccade patterns. In 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (pp. 647-650). IEEE. https://doi.org/10.1109/ICMEW.2019.00125
[11]Wloka, C., Kotseruba, I., & Tsotsos, J. K. (2017). Saccade sequence prediction: Beyond static saliency maps. arXiv preprint arXiv:1711.10959. https://doi.org/10.48550/arXiv.1711.10959
[12]Halim Abbas, Ford Garberson, Eric Glover, Dennis P Wall, Machine learning approach for early detection of autism by combining questionnaire and home video screening, Journal of the American Medical Informatics Association, Volume 25, Issue 8, August 2018, Pages 1000–1007, https://doi.org/10.1093/jamia/ocy039
[13]Tariq, Q., Daniels, J., Schwartz, J. N., Washington, P., Kalantarian, H., & Wall, D. P. (2018). Mobile detection of autism through machine learning on home video: A development and prospective validation study. PLoS medicine, 15(11), e1002705.
[14]Hosseini, M. P., Beary, M., Hadsell, A., Messersmith, R., & Soltanian-Zadeh, H. (2021). Deep Learning for Autism Diagnosis and Facial Analysis in Children. Frontiers in Computational Neuroscience, 15. https://doi.org/10.3389/fncom.2021.789998
[15]Y. Khosla, P. Ramachandra and N. Chaitra. (2021) . Detection of autistic individuals using facial images and deep learning.IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS), 2021, pp. 1-5, https://doi.org/10.1109/CSITSS54238.2021.9683205
[16]Wang, X., & Yu, H. (2005, May). How to break MD5 and other hash functions. In Annual international conference on the theory and applications of cryptographic techniques (pp. 19-35). Springer, Berlin, Heidelberg. https://doi.org/10.1007/11426639_2
[17]Lu, A., & Perkowski, M. (2021). Deep Learning Approach for Screening Autism Spectrum Disorder in Children with Facial Images and Analysis of Ethnoracial Factors in Model Development and Application. Brain Sciences, 11(11), 1446.
[18]Liao, D., & Lu, H. (2018, March). Classify autism and control based on deep learning and community structure on resting-state fMRI. In 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI) (pp. 289-294). IEEE. https://doi.org/10.1109/ICACI.2018.8377471
[19]Welcome to the Autism Brain Imaging Data Exchange!, Child Mind Institute, https://fcon_1000.projects.nitrc.org/indi/abide/. Accessed 26 August 2022
[20]Liu, Y., Xu, L., Li, J., Yu, J., & Yu, X. (2020). Attentional connectivity-based prediction of autism using heterogeneous rs-fMRI data from CC200 atlas. Experimental Neurobiology, 29(1), 27.https://doi.org/10.5607/en.2020.29.1.27
[21]Kojovic, N., Natraj, S., Mohanty, S.P. et al. Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Sci Rep 11, 15069 (2021). https://doi.org/10.1038/s41598-021-94378-z
[22]Analytics Vidya, Introduction to Convolution Neural Network(CNN), https://www.analyticsvidhya.com/blog/2021/05/convolutional-neural-networks-cnn/
[23]Machine Learning Mastery, A Gentle Introduction to Long-Short Term Memory Networks by the Experts, https://machinelearningmastery.com/gentle-introduction-long-short-term-memory-networks-experts/
[24]Felix A. Gers, Jürgen Schmidhuber, Fred Cummins; Learning to Forget: Continual Prediction with LSTM. Neural Comput 2000; 12 (10): 2451–2471. doi: https://doi.org/10.1162/089976600300015015
[25]A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke and J. Schmidhuber, "A Novel Connectionist System for Unconstrained Handwriting Recognition," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855-868, May 2009, doi: 10.1109/TPAMI.2008.137.
[26]Analytics Vidya, Learn About Long-Short Term Memory (LSTM) Algorithms, https://www.analyticsvidhya.com/blog/2021/03/introduction-to-long-short-term-memory-lstm/
[27]Paperswithcode, Bidirectional LSTM, https://paperswithcode.com/method/bilstm
[28]Sample YouTube Video: https://www.youtube.com/watchv=pSGVb60-BSw Accessed 26 January 2023
[29]Pulli, K., Baksheev, A., Kornyakov, K., & Eruhimov, V. (2012). Real-time computer vision with OpenCV. Communications of the ACM, 55(6), 61-69. https://doi.org/10.1145/2184319.2184337
[30]Western Psychological Services. Schopler, E., Reichler, R. J., & Renner, B. R. (1988). Childhood Autism Rating Scale (CARS). Western Psychological Services.
[31]Tammina, S. (2019). Transfer learning using vgg-16 with deep convolutional neural network for classifying images. International Journal of Scientific and Research Publications (IJSRP), 9(10), 143-150. http://dx.doi.org/10.29322/IJSRP.9.10.2019.p9420
[32]Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826).https://doi.org/10.48550/arXiv.1512.00567
[33]Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258).https://doi.org/10.48550/arXiv.1610.02357
[34]Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.https://doi.org/10.48550/arXiv.1704.04861
[35]Viso.ai, The Complete Guide to OpenPose in 2023 , https://viso.ai/deep-learning/openpose/#:~:text=OpenPose%20is%20a%20real%2Dtime,a%20total%20of%20135%20keypoints
[36]FFmpeg, ffmpeg Documentation, https://ffmpeg.org/ffmpeg.html