With the expansion of worldwide security concerns and a consistently expanding requirement for successful checking of open places, i.e. air terminals, railroad stations, shopping centres, crowded sports fields, army bases or smart healthcare facilities such as daily activity monitoring and fall detection in old people’s homes is increasing very rapidly. The visual occlusions and ambiguities in crowded scenes, usage of suitable method and in addition the perplexing practices and scene semantics make the investigation a challenging task. This research demonstrates comprehensive and critical analysis of crowd scene involves in object detection, tracking, feature extraction and learning from visual surveillance which helps to recognize behavioural pattern. This research refers scene understanding as scene layout, i.e. finding streets, structures, side-walks, vehicles turning, person on foot intersection and scene status such as crowd congestion, split, merge etc. The  significance of the proposed comprehensive review to create crowd administration procedures and help the development of the group or people, to maintain a strategic distance from the group calamities and guarantee general society security. Based on the observation of previous research in three aspects, i.e. review based on methods, frameworks and critical existing results analysis, this research propose a framework for anomaly detection in crowded scene using LSTM (long Short-Term Method).  Proposed comprehensive review is expected to contribute significantly for the investigation of behavior pattern analysis in computer vision research domains.