Ayman H. Tanira; Wesam M. Ashour

A Hybrid Unsupervised Density-based Approach with Mutual Information for Text Outlier Detection

Full Text (PDF, 746KB), PP.41-56

Views: 0 Downloads: 0

Author(s)

Ayman H. Tanira ^1,* Wesam M. Ashour ²

1. Computer Science Department, Palestine Technical College, Deir El-Balah, Palestine

2. Computer Engineering, Islamic University of Gaza, Gaza, Palestine

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2023.05.04

Received: 19 Jun. 2023 / Revised: 3 Aug. 2023 / Accepted: 27 Aug. 2023 / Published: 8 Oct. 2023

Index Terms

Text Mining, Text Outliers, Density-based, Mutual Information

Abstract

The detection of outliers in text documents is a highly challenging task, primarily due to the unstructured nature of documents and the curse of dimensionality. Text document outliers refer to text data that deviates from the text found in other documents belonging to the same category. Mining text document outliers has wide applications in various domains, including spam email identification, digital libraries, medical archives, enhancing the performance of web search engines, and cleaning corpora used in document classification. To address the issue of dimensionality, it is crucial to employ feature selection techniques that reduce the large number of features without compromising their representativeness of the domain. In this paper, we propose a hybrid density-based approach that incorporates mutual information for text document outlier detection. The proposed approach utilizes normalized mutual information to identify the most distinct features that characterize the target domain. Subsequently, we customize the well-known density-based local outlier factor algorithm to suit text document datasets. To evaluate the effectiveness of the proposed approach, we conduct experiments on synthetic and real datasets comprising twelve high-dimensional datasets. The results demonstrate that the proposed approach consistently outperforms conventional methods, achieving an average improvement of 5.73% in terms of the AUC metric. These findings highlight the remarkable enhancements achieved by leveraging normalized mutual information in conjunction with a density-based algorithm, particularly in high-dimensional datasets.

Cite This Paper

Ayman H. Tanira, Wesam M. Ashour, "A Hybrid Unsupervised Density-based Approach with Mutual Information for Text Outlier Detection", International Journal of Intelligent Systems and Applications(IJISA), Vol.15, No.5, pp.41-56, 2023. DOI:10.5815/ijisa.2023.05.04

Reference

[1]M. Agyemang, K. Barker, and R. Alhajj, “Web outlier mining: Discovering outliers from web datasets,” in Intelligent Data Analysis, 2005. doi: 10.3233/ida-2005-9505.
[2]“IBM Blog.” https://www.ibm.com/blog/ (accessed Jul. 08, 2023).
[3]R. Kannan, H. Woo, C. C. Aggarwal, and H. Park, “Outlier detection for text data,” Proc West Mark Ed Assoc Conf, pp. 489–497, 2017, doi: 10.1137/1.9781611974973.55.
[4]W. A. Mohotti and R. Nayak, “Efficient Outlier Detection in Text Corpus Using Rare Frequency and Ranking,” ACM Trans Knowl Discov Data, vol. 14, no. 6, 2020, doi: 10.1145/3399712.
[5]M. Agyemang, K. Barker, and R. S. Alhajj, “WCOND-mine: Algorithm for detecting web content outliers from web documents,” in Proceedings - IEEE Symposium on Computers and Communications, 2005. doi: 10.1109/ISCC.2005.155.
[6]C. C. Aggarwal and C. X. Zhai, Mining text data, vol. 9781461432234. 2013. doi: 10.1007/978-1-4614-3223-4.
[7]J. Zhang, X. Long, and T. Suel, “Performance of compressed inverted list caching in search engines,” in Proceeding of the 17th International Conference on World Wide Web 2008, WWW’08, 2008. doi: 10.1145/1367497.1367550.
[8]A. Hotho, A. Nürnberger, and G. Paaß, “A Brief Survey of Text Mining,” Journal for Language Technology and Computational Linguistics, vol. 20, no. 1, 2005, doi: 10.21248/jlcl.20.2005.68.
[9]D. Chakraborty, V. Narayanan, and A. Ghosh, “Integration of deep feature extraction and ensemble learning for outlier detection,” Pattern Recognit, vol. 89, 2019, doi: 10.1016/j.patcog.2019.01.002.
[10]Y. Liu et al., “Generative Adversarial Active Learning for Unsupervised Outlier Detection,” IEEE Trans Knowl Data Eng, vol. 32, no. 8, 2020, doi: 10.1109/TKDE.2019.2905606.
[11]K. E. and N. R., “Algorithms for Mining Distance-Based Outliers in Large Datasets,” in Proceedings of the VLDB Conference, New York, USA, 1998.
[12]S. D. Bay and M. Schwabacher, “Mining distance-based outliers in near linear time with randomization and a simple pruning rule,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2003. doi: 10.1145/956750.956758.
[13]M. I. Petrovskiy, “Outlier detection algorithms in data mining systems,” Programming and Computer Software, vol. 29, no. 4, pp. 228–237, 2003, doi: 10.1023/A:1024974810270/METRICS.
[14]E. Fouche, Y. Meng, F. Guo, H. Zhuang, K. Bohm, and J. Han, “Mining text outliers in document directories,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2020. doi: 10.1109/ICDM50108.2020.00024.
[15]F. Cao, X. Wu, L. Yu, and J. Liang, “An outlier detection algorithm for categorical matrix-object data,” Appl Soft Comput, vol. 104, 2021, doi: 10.1016/j.asoc.2021.107182.
[16]M. Koppel and S. Seidman, “Detecting pseudepigraphic texts using novel similarity measures,” Digital Scholarship in the Humanities, vol. 33, no. 1, 2018, doi: 10.1093/llc/fqx011.
[17]M. M. Breuniq, H. P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying density-based local outliers,” SIGMOD Record (ACM Special Interest Group on Management of Data), vol. 29, no. 2, 2000, doi: 10.1145/335191.335388.
[18]A. H. Tanira, A. A. Rafea, and H. A. Hassan, “A density-based approach wiith vsm and weighted n-grams for detecting web document outliers,” in Proceedings of the ISCA 24th International Conference on Computer Applications in Industry and Engineering, CAINE 2011, 2011.
[19]B. Tang and H. He, “A local density-based approach for outlier detection,” Neurocomputing, vol. 241, 2017, doi: 10.1016/j.neucom.2017.02.039.
[20]J. Ning, L. Chen, and J. Chen, “Relative density-based outlier detection algorithm,” in ACM International Conference Proceeding Series, 2018. doi: 10.1145/3297156.3297236.
[21]L. Duan, L. Xu, Y. Liu, and J. Lee, “Cluster-based outlier detection,” Ann Oper Res, vol. 168, no. 1, 2009, doi: 10.1007/s10479-008-0371-9.
[22]W. Baoyi, L. Xiangyu, and Z. Shaomin, “An improved outlier detection algorithm K-LOF based on density,” Computing, Performance and Communication Systems, vol. 2, no. 1, pp. 1–7, Dec. 2017, doi: 10.23977/CPCS.2017.21001.
[23]F. Lazhar, “Fuzzy clustering-based semi-supervised approach for outlier detection in big text data,” Progress in Artificial Intelligence, vol. 8, no. 1, 2019, doi: 10.1007/s13748-018-0165-5.
[24]M. Mohaghegh and A. Abdurakhmanov, “Anomaly Detection in Text Data Sets using Character-Level Representation,” in Journal of Physics: Conference Series, 2021. doi: 10.1088/1742-6596/1880/1/012028.
[25]R. Battiti, “Using Mutual Information for Selecting Features in Supervised Neural Net Learning,” IEEE Trans Neural Netw, vol. 5, no. 4, 1994, doi: 10.1109/72.298224.
[26]H. Peng, F. Long, and C. Ding, “Feature selection based on mutual information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy,” IEEE Trans Pattern Anal Mach Intell, vol. 27, no. 8, 2005, doi: 10.1109/TPAMI.2005.159.
[27]P. A. Estévez, M. Tesmer, C. A. Perez, and J. M. Zurada, “Normalized mutual information feature selection,” IEEE Trans Neural Netw, vol. 20, no. 2, 2009, doi: 10.1109/TNN.2008.2005601.
[28]K. Zhou, W. Wang, T. Hu, and K. Deng, “Application of improved asynchronous advantage actor critic reinforcement learning model on anomaly detection,” Entropy, vol. 23, no. 3, 2021, doi: 10.3390/e23030274.
[29]D. Zha, K. H. Lai, M. Wan, and X. Hu, “Meta-AAD: Active anomaly detection with deep reinforcement learning,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2020. doi: 10.1109/ICDM50108.2020.00086.
[30]G. Pang, A. Van Den Hengel, C. Shen, and L. Cao, “Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2021. doi: 10.1145/3447548.3467417.
[31]M. Yu and S. Sun, “Policy-based reinforcement learning for time series anomaly detection,” Eng Appl Artif Intell, vol. 95, 2020, doi: 10.1016/j.engappai.2020.103919.
[32]S. M. Weiss, N. Indurkhya, T. Zhang, and F. J. Damerau, Text mining: Predictive methods for analyzing unstructured information. 2005. doi: 10.1007/978-0-387-34555-0.
[33]“(10)Dataset Text Document Classification | Kaggle.” https://www.kaggle.com/datasets/jensenbaxter/10dataset-text-document-classification (accessed Jul. 08, 2023).
[34]“Reuters-21578 Text Categorization Collection - UCI Machine Learning Repository.” https://archive.ics.uci.edu/dataset/137/reuters+21578+text+categorization+collection (accessed Jul. 08, 2023).
[35]C. Vidyadhari, N. Sandhya, and P. Premchand, “Automatic Incremental Clustering Using Bat-Grey Wolf Optimizer-Based MapReduce Framework for Effective Management of High-Dimensional Data,” International Journal of Ambient Computing and Intelligence, vol. 11, no. 4, 2020, doi: 10.4018/IJACI.2020100105.

International Journal of Intelligent Systems and Applications (IJISA)