Abdiansah Abdiansah; Azhari Azhari; Anny K. Sari

Survey on Answer Validation for Indonesian Question Answering System (IQAS)

Full Text (PDF, 421KB), PP.68-78

Views: 0 Downloads: 0

Author(s)

Abdiansah Abdiansah ^1,* Azhari Azhari ² Anny K. Sari ²

1. Universitas Sriwijaya, South-Sumatera, 30662, Indonesia

2. Universitas Gadjah Mada, Yogyakarta, 55281, Indonesia

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2018.04.08

Received: 15 Apr. 2017 / Revised: 11 Jul. 2017 / Accepted: 15 Sep. 2017 / Published: 8 Apr. 2018

Index Terms

Question Answering System, Indonesian Question Answering System, Answer Validation, QAS, IQAS, AV

Abstract

Research on Question Answering System (QAS) has been done mainly in English. Unfortunately, for Indonesian, it is still rarely explored whereas Indonesian is the official language used more than 250 million people. Research in the area of Indonesian Question Answering System (IQAS) began in 2005s, and since then only few number of IQAS have been developed. One of the important issues in IQAS is Answer Validation (AV), which is a system that can determine the correctness of QAS. To identify the future scope of research in this area, the need of comprehensive survey on IQAS and AV arises naturally. The goals of this survey are to find the cutting-edge method used in AV and to prove that AV has not been implemented on IQAS. Based on the results, we suggest new opportunities and research challenges for IQAS community.

Cite This Paper

Abdiansah Abdiansah, Azhari Azhari, Anny K. Sari, "Survey on Answer Validation for Indonesian Question Answering System (IQAS)", International Journal of Intelligent Systems and Applications(IJISA), Vol.10, No.4, pp.68-78, 2018. DOI:10.5815/ijisa.2018.04.08

Reference

[1]S. Büttcher, C. L. Clarke, and G. V. Cormack, Information retrieval: Implementing and evaluating search engines. MIT Press, 2016. doi:10.1108/02640471111188088.
[2]S. K. Dwivedi and V. Singh, “Research and Reviews in Question Answering System,” in Procedia Technology, 2013, vol. 10, pp. 417–424. doi: 10.1016/ j.protcy.2013.12.378.
[3]A. Mishra and S. K. Jain, “A survey on question answering systems with classification,” J. King Saud Univ. - Comput. Inf. Sci., vol. 28, no. 3, pp. 345–361, 2016. doi:10.1016/j.jksuci.2014.10.007.
[4]A. Mathur and M. T. U. Haider, “Question answering system: A survey,” in 2015 International Conference on Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials, ICSTM 2015 - Proceedings, 2015, no. May, pp. 47–57. doi:10.1109/ICSTM.2015.7225389.
[5]Á. Rodrigo and A. Peñas, “On Evaluating the Contribution of Validation for Question Answering,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 4, pp. 1157–1161, 2015. doi:10.1109/TKDE.2014.2373363.
[6]A. Penas, Á. Rodrigo, V. Sama, and F. Verdejo, “Overview of the Answer Validation Exercise 2006,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5706 LNCS, 2007, pp. 257–264. doi:10.1007/978-3-540-74999-8_32.
[7]A. Penas, Á. Rodrigo, and F. Verdejo, “Overview of the Answer Validation Exercise 2007,” in Advances in Multilingual and Multimodal Information Retrieval, vol. 5706 LNCS, Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 237–248. doi:10.1007/978-3-540-85760-0_28.
[8]Á. Rodrigo, A. Peñas, and F. Verdejo, “Overview of the answer validation exercise 2008,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5706 LNCS, pp. 296–313, 2009. doi:10.1007/978-3-642-04447-2_35.
[9]I. Dagan, D. Roth, M. Sammons, and F. Zanzotto, “Recognizing textual entailment: Models and applications,” in Synthesis Lectures on Human Language Technologies, Morgan & Claypool, 2013, pp. 1–220. doi:10.2200/S00509ED1V01Y201305HLT023.
[10]S. Babych, A. Henn, J. Pawellek, and S. Pado, “Dependency-based answer validation for German,” in CEUR Workshop Proceedings, 2011, vol. 1177.
[11]H. Gómez-Adorno, D. Pinto, and D. Vilariño, “A Question Answering System for Reading Comprehension Tests,” in 5th Mexican Conference, MCPR 2013, 2013, vol. 1087, pp. 354–363. doi:10.1007/978-3-642-38989-4_36.
[12]Á. Rodrigo, J. Perez-Iglesias, A. Peñas, G. Garrido, and L. Araujo, “A Question Answering System based on Information Retrieval and Validation,” in Notebook Paper for the CLEF 2010 LABs Workshop, 2010, pp. 22–23. doi:10.1.1.175.1640.
[13]A. Grappy, B. Grau, M. H. Falco, A. L. Ligozat, I. Robba, and A. Vilnat, “Selecting answers to questions from Web documents by a robust validation process,” in Proceedings - 2011 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2011, 2011, vol. 1, pp. 55–62. doi:10.1109/WI-IAT.2011.210.
[14]R. F. Simmons, “Answering English questions by computer: a survey,” Commun. ACM, vol. 8, no. 1, pp. 53–70, Jan. 1965. doi:10.1145/363707.363732.
[15]M. Adriani and Rinawati, “University of indonesia participation at query answering-CLEF 2005,” in CEUR Workshop Proceedings, 2005, vol. 1171, pp. 4–6.
[16]S. H. Wijono, I. Budi, L. Fitria, and M. Adriani, “Finding answers to Indonesian questions from English documents,” in CEUR Workshop Proceedings, 2006, vol. 1172, pp. 1–4.
[17]M. Adriani and S. Adiwibowo, “Finding answers using resources in the internet,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5152 LNCS, 2008, pp. 332–335. doi:10.1007/978-3-540-85760-0-41.
[18]S. D. Larasati and R. Manurung, “Towards a semantic analysis of bahasa Indonesia for question answering,” in Proceedings of the 10th Conference of the Pacific Association for Computational Linguistics, 2007, no. November, pp. 273–280. http://staf.cs.ui.ac.id/~maruli/ pub/pacling07.pdf.
[19]A. Purwarianti, M. Tsuchiya, and S. Nakagawa, “A Machine Learning Approach for Indonesian Question Answering System,” in Proceedings of the 25th Conference on Proceedings of the 25th IASTED International Multi-Conference: Artificial Intelligence and Applications, 2007, pp. 537–542. http://dl.acm.org/ citation.cfm?id=1295303.1295395.
[20]R. Mahendra, S. D. Larasati, R. Manurung, and M. De La Salle Univ, “Extending an Indonesian Semantic Analysis-based Question Answering System with Linguistic and World Knowledge Axioms,” in Paclic 22: Proceedings of the 22nd Pacific Asia Conference on Language, Information and Computation, 2008, pp. 262–271. doi:10.1.1.210.120.
[21]H. Toba and M. Adriani, “Pattern Based Indonesian Question Answering System,” in Proceedings of the International Conference on Advanced Computer Systems and Information Systems (ICACSIS) University of Indonesia, 2009.
[22]M. Iqbal Faruqi and A. Purwarianti, “An Indonesian question analyzer to enhance the performance of Indonesian-English CLQA,” in Proceedings of the 2011 International Conference on Electrical Engineering and Informatics, ICEEI 2011, 2011, no. July. doi:10.1109/ICEEI.2011.6021513.
[23]A. Purwarianti and N. Yusliani, “Sistem Question Answering Bahasa Indonesia untuk Pertanyaan Non-Factoid,” J. Ilmu Komput. dan Inf., vol. 4, no. 1, p. 10, May 2012. doi:10.21609/jiki.v4i1.151.
[24]A. A. Zulen and A. Purwarianti, “Study and implementation of monolingual approach on indonesian question answering for factoid and non-factoid question,” in PACLIC 25 - Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, 2011, pp. 622–631. http://www.scopus.com/inward/ record.url?eid=2-s2.0-84863874779&partnerID=40&md5= cfbc9e0618d9d103b62f31b3c184750a.
[25]A. Fikri and A. Purwarianti, “Case based Indonesian closed domain question answering system with real world questions,” in 7th International Conference on Telecommunication Systems, Services, and Applications, TSSA 2012, 2012, pp. 181–186. doi:10.1109/TSSA.2012.6366047.
[26]R. H. Gusmita, Y. Durachman, S. Harun, A. F. Firmansyah, H. T. Sukmana, and A. Suhaimi, “A rule-based question answering system on relevant documents of Indonesian Quran Translation,” in International Conference on Cyber and IT Service Management (CITSM), 2014, pp. 104–107. doi: 10.1109/ CITSM.2014.7042185.
[27]W. Suwarningsih, I. Supriana, and A. Purwarianti, “Discovery indonesian medical question-answering pairs pattern with question generation,” Int. J. Appl. Eng. Res., vol. 10, no. 14, pp. 34217–34223, 2015.
[28]M. Z. Naf’an, D. E. Mahmudah, S. J. Putra, and A. F. Firmansyah, “Eliminating unanswered questions from question answering system for Khulafaa Al-Rashidin history,” Proc. - 6th Int. Conf. Inf. Commun. Technol. Muslim World, ICT4M 2016, pp. 140–143, 2016. doi:10.1109/ICT4M.2016.33.
[29]W. Suwarningsih, A. Purwarianti, and I. Supriana, “Reducing the Conflict Factors Strategies in Question Answering System,” in IOP Conference Series: Materials Science and Engineering, 2017, vol. 180, pp. 12–20. doi:10.1088/1742-6596/755/1/011001.
[30]A. Solovyev, “Dependency-Based Algorithms for Answer Validation Task in Russian Question Answering,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8105 LNAI, 2013, pp. 199–212. doi:10.1007/978-3-642-40722-2_20.
[31]I. Nikolova, I. Zamanov, M. Kraeva, N. Hateva, I. Yovcheva, and G. Angelova, “Voltron: A Hybrid System For Answer Validation Based On Lexical And Distance Features,” in Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), 2015, pp. 242–246.
[32]V. Nguyen and A. Le, “Answer Validation for Question Answering Systems by Using External Resources,” in Integrated Uncertainty in Knowledge Modelling and Decision Making (Lecture Notes in Computer Science), Springer Berlin Heidelberg, 2016, pp. 305–316. doi:10.1007/978-3-319-49046-5_26.
[33]P. Ranjan and R. C. Balabantaray, “Question answering system for factoid based question,” in Contemporary Computing and Informatics (IC3I), 2016, pp. 221–224. doi:10.1109/IC3I.2016.7917964.
[34]N. Indurkhya and F. J. Damerau, Handbook of Natural Language Processing, 2nd ed. 2010. doi:10.1162/COLI_r_00048.
[35]S. Shekarpour, E. Marx, S. Auer, and A. P. Sheth, “RQUERY: Rewriting Natural Language Queries on Knowledge Graphs to Alleviate the Vocabulary Mismatch Problem,” AAAI, pp. 3936–3943, 2017.
[36]E. M. Voorhees, “Overview of the TREC 2002 question answering track,” in Proceedings of the 11th Text Retrieval Conference, 2002, p. 1.
[37]M. Tatu, B. Iles, and D. Moldovan, “Automatic Answer Validation Using COGEX,” in 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, 2007, vol. 1172, pp. 494–501. doi:10.1007/978-3-540-74999-8_59.
[38]Y. Zhang and D. Zhang, “Enabling answer validation by logic form reasoning in Chinese question answering,” in NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings, 2003, vol. 10, pp. 275–280. doi:10.1109/NLPKE.2003.1275912.
[39]Dongfeng Cai, Yanju Dong, D. Lv, Guiping Zhang, and Xuelei Miao, “A Web-Based Chinese Question Answering with Answering Validation,” in 2005 International Conference on Natural Language Processing and Knowledge Engineering, 2005, vol. 1, pp. 499–502. doi:10.1109/NLPKE.2005.1598788.
[40]M. Kouylekov and B. Magnini, “Recognizing textual entailment with tree edit distance algorithms,” in PASCAL Challenges on RTE, 2006, pp. 17–20. http:// citeseerx.ist.psu.edu/viewdoc/summary?doi=?doi=10.1.1.124.247.
[41]I. Glöckner and K. H. Weis, “An integrated machine learning and case-based reasoning approach to answer validation,” in Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012, 2012, vol. 1, pp. 494–499. doi:10.1109/ICMLA.2012.90.
[42]S. K. Ray, S. Singh, and B. P. Joshi, “World Wide Web based Question Answering System - a relevance feedback framework for automatic answer validation,” in 2009 Second International Conference on the Applications of Digital Information and Web Technologies, 2009, pp. 169–174. 10.1109/ICADIWT.2009.5273942.
[43]A. L. Ligozat, B. Grau, A. Vilnat, I. Robba, and A. Grappy, “Towards an automatic validation of answers in question answering,” in Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI, 2007, vol. 2, pp. 444–447. doi:10.1109/ICTAI.2007.156.
[44]Á. Rodrigo, A. Peñas, J. Herrera, and F. Verdejo, “The Effect of Entity Recognition on Answer Validation,” in 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain, September 20-22, 2006, 2007, pp. 483–489. doi:10.1007/978-3-540-74999-8_57.
[45]O. Ferrandez, R. Munoz, and M. Palomar, “TE4AV: Textual Entailment for Answer Validation,” in 2008 International Conference on Natural Language Processing and Knowledge Engineering, 2008, pp. 1–8. doi:10.1109/NLPKE.2008.4906746.
[46]A. Tellez-Valero, M. Montes-y-Gómez, L. Villaseñor-Pineda, and A. Peñas, “Improving Question Answering by Combining Multiple Systems Via Answer Validation,” in Computational Linguistics and Intelligent Text Processing, vol. 4919 LNCS, Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 544–554. doi:10.1007/978-3-540-78135-6_47.
[47]A. Tellez-Valero, M. Montes-y-Gómez, and L. Villaseñor-Pineda, “Towards multi-stream question answering using answer validation,” Inform., vol. 34, no. 1, pp. 45–54, 2010. doi:10.1.1.61.1041.
[48]P. Pakray, S. Pal, S. Bandyopadhyay, and A. Gelbukh, “Automatic Answer Validation System on English language,” in 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE), 2010, vol. 6, pp. V6-329-V6-333. doi:10.1109/ ICACTE.2010.5579166.
[49]P. Pakray, U. Barman, S. Bandyopadhyay, and A. Gelbukh, “Semantic Answer Validation using Universal Networking Language,” Int. J. Comput. Sci. Inf. Technol., vol. 3, no. 4, pp. 4927–4932, 2012. doi:10.1.1.259.6069.
[50]Sharma and K. Sudhir, “Sentiment Predictions using Support Vector Machines for Odd-Even Formula in Delhi,” Int. J. Intell. Syst. Appl., vol. 9, no. July, pp. 61–69, 2017. doi:10.5815/ijisa.2017.07.07.
[51]A. E. Khedr, “Predicting Stock Market Behavior using Data Mining Technique and News Sentiment Analysis,” Int. J. Intell. Syst. Appl., vol. 9, no. July, pp. 22–30, 2017. doi:10.5815/ijisa.2017.07.03.
[52]I. S. I. Abuhaiba and H. M. Dawoud, “Combining Different Approaches to Improve Arabic Text Documents Classification,” Int. J. Intell. Syst. Appl., vol. 9, no. April, pp. 39–52, Apr. 2017. doi:10.5815/ijisa.2017.04.05.

International Journal of Intelligent Systems and Applications (IJISA)