Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by difficulty in social interactions, communication, and repetitive behaviors. Protocols like ADOS (Autism Diagnostic Observation Schedule) and ADI-R (Autism Diagnostic Interview Revised) are used by experts to assess the subject’s behavior which is time-consuming. Over the decade, researchers have studied the application of various Machine Learning techniques for ASD diagnosis through facial feature analysis, eye movement tracking, questionnaire analysis, functional magnetic resonance imaging (fMRI) analysis, etc. However, these techniques are not helpful for the parent or guardian of the child to perform an initial screening. This research proposes a novel deep learning model to diagnose ASD using general videos of the subject performing some tasks with the parent/guardian. Since there is no publicly available dataset on ASD videos, a dataset is created by collecting the videos of autistic children performing some activities with parents/guardians from YouTube from different demographic locations. These videos are then converted to skeletal key points to extract the child's engagement and social interaction in a given task. The proposed CNN-LSTM model is trained on 80% of the collected videos and then tested on the remaining 20%. The experiment results on various combinations of pre-trained CNN models and LSTM/BiLSTM show that the proposed model can be used as an initial autism screening tool. Among the different combinations, the MobileNet and Bi-LSTM combo achieved the best test accuracy of 84.95% with 89% precision, recall and F1-score.