International Journal of Information Technology and Computer Science (IJITCS)
IJITCS Vol. 8, No. 8, 8 Aug. 2016
Cover page and Table of Contents: PDF (size: 317KB)
Multi Objective Test Suite Reduction for GUI Based Software Using NSGA-II
Full Text (PDF, 317KB), PP.59-65
Views: 0 Downloads: 0
Author(s)
Neha Chaudhary
1,*
O.P. Sangwan
2
Index Terms
Test Suite Reduction, NSGA II, Multi Objective Optimization, Pareto-optimal solution
Abstract
Regression Testing is a performed to ensure modified code does not have any unintended side effect on the software. If regression testing is performed with retest-all method it will be very time consuming as testing activity. Therefore test suite reduction methods are used to reduce the size of original test suite. Objective of test suite reduction is to reduce those test cases which are redundant or less important in their fault revealing capability. Test suite reduction can only be used when time is critical to run all test cases and selective testing can only be done. Various methods exist in the literature related to test suite reduction of traditional software. Most of the methods are based of single objective optimization. In case of multi objective optimization of test suite, usually researchers assign different weight values to different objectives and combine them as single objective. However in test suite reduction multiple Pareto-optimal solutions are present, it is difficult to select one test case over other. Since GUI based software is our concern there exist very few reduction techniques and none of them consider multiple objective based reduction. In this work we propose a new test suite reduction technique based on two objectives, event weight and number of faults identified by test case. We evaluated our results for 2 different applications and we achieved 20% reduction in test suite size for both applications. In Terp Paint 3.0 application compromise 15.6% fault revealing capability and for Notepad 11.1% fault revealing capability is reduced.
Cite This Paper
Neha Chaudhary, O.P. Sangwan, "Multi Objective Test Suite Reduction for GUI Based Software Using NSGA-II", International Journal of Information Technology and Computer Science(IJITCS), Vol.8, No.8, pp.59-65, 2016. DOI:10.5815/ijitcs.2016.08.07
Reference
[1]Memon Atif, Lou Soffa Mary, E. Pollock Martha, “Coverage criteria for GUI testing”, in the proceeding of 21st International conference on software engineering, ACM press, pp 257-266, 1999.
[2]Neha Chaudhary, O.P. Sangwan, Richa Arora,” Event-Coverage and Weight based Method for Test Suite Prioritization”, I.J. Information Technology and Computer Science, Vol. 12, pp. 61-66, 2014.
[3]Reference: Terp Paint 3.0 Fault matrix http://www.cs.umd.edu/~atif/Benchmarks/UMD2007a.html
[4]Kalyanmoy Deb and Deb Kalyanmoy, “Multi-Objective Optimization Using Evolutionary Algorithms” John Wiley & Sons, Inc., New York, NY, USA, 2001.
[5]Kaur and Kumari, HP QuickTest Professional version 11. 2010. HP – QTP version 11, Comparative study of Automated Testing Tools: Test Complete and QuickTest Pro, Punjab University, 2011.
[6]Paul Gerrard, “Testing GUI Applications”, EuroSTAR, Edinburgh UK, 1997
[7]A. M. Atif, “Automatically Repairing Event Sequence-Based GUI Test Suites for Regression Testing”, ACM Transaction on Software Engineering and Method. Volume 18, Issue 2, Nov. 2008.
[8]Kalyanmoy Deb, “Multi-Objective Optimization Using Evolutionary Algorithms”, Wiley India Private Limited, ISBN-10: 8126528044.
[9]McMaster, S.; Memon, A.M., "Call-Stack Coverage for GUI Test Suite Reduction," Software Engineering, IEEE Transactions on Software Engineering, vol.34, no.1, pp.99-115, Jan.-Feb. 2008
[10]Preethi Harris and Nedunchezhian Raju,” A Greedy Approach for Coverage-Based Test Suite”, The International Arab Journal of Information Technology, Vol. 12, No.1, PP. 17-23, January 2015.
[11]M. Jean Harrold, Rajiv Gupta, and Mary Lou Soffa. 1993. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2, pp. 270-285, July 1993.
[12]Arlt, Stephan and Podelski, Andreas and Wehrle, Martin, ”Reducing GUI Test Suites via Program Slicing”, Proceedings of the 2014 International Symposium on Software Testing and Analysis, 2014, pp.270-281, 2014.
[13]G. Rothermel, M. J. Harrold, J. von Ronne, and C. Hong, ”Empirical studies of test-suite reduction”, Journal of Software Testing, Verification, and Reliability, Vol 12, no. 4, December, 2002.
[14]G. Rothermel, R. Untch, C. Chu, and M. J. Harrold, ”Test case prioritization”, IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929-948, October, 2001.
[15]Shin Yoo, Mark Harman, and Shmuel Ur, ”Highly scalable multi objective test suite minimisation using graphics cards”, Proceedings of the Third international conference on Search based software engineering, pp. 219-236, 2011.
[16]Shin Yoo and Mark Harman, “Pareto efficient multi-objective test case selection”, In Proceedings of the 2007 international symposium on Software testing and analysis (ISSTA '07), ACM, New York, NY, USA, pp. 140-150, 2007.
[17]Saeed Parsa and Alireza Khalilian,” On the Optimization Approach towards Test Suite Minimization”, International Journal of Software Engineering and Its Applications Vol. 4, No. 1, January 2010.
[18]M. R. Garey and D. S. Johnson, “Computers and Intractability: A guide to the theory of NP-Completeness”, New York, NY: W. H. Freeman and Company, 1979.
[19]J. O_utt, J. Pan, and J. Voas, “Procedures for reducing the size of coverage-based test sets," in Proceedings of the 12th International Conference on Testing Computer Software. ACM Press, pp. 111-123, June 1995.
[20]T. Y. Chen and M. F. Lau, “Dividing strategies for the optimization of a test suite," Information Processing Letters, vol. 60, no. 3, pp. 135-141, 1996.
[21]W. E. Wong, J. R. Horgan, S. London, and A. P. Mathur, “Effect of test set minimization on fault detection effectiveness," Software Practice and Experience, vol. 28, no. 4, pp. 347-369, April 1998.
[22]W. E. Wong, J. R. Horgan, A. P. Mathur, and A. Pasquini, “Test set size minimization and fault detection effectiveness: A case study in a space application," The Journal of Systems and Software, vol. 48, no. 2, pp. 79-89, October 1999.
[23]G. Rothermel, S. Elbaum, A. Malishevsky, P. Kallakuri, and B. Davia, “The impact of test suite granularity on the cost-effectiveness of regression testing," in Proceedings of the 24th International Conference on Software Engineering (ICSE 2002). ACM Press, pp. 130-140, May 2002.
[24]S. Yoo and M. Harman, “Pareto efficient multi-objective test case selection," in Proceedings of International Symposium on Software Testing and Analysis. ACM Press, pp. 140-150, July 2007.
[25]M. Jean Harrold, Rajiv Gupta, and Mary Lou Soffa, “A methodology for controlling the size of a test suite” ACM Trans. Softw. Eng. Methodol. 2, 270-285, 1993.
[26]Jun-Wei Lin and Chin-Yu Huang, “Analysis of test suite reduction with enhanced tie-breaking techniques”, Inf. Softw. Technol. 51, 679-690, 2009.
[27]Saeed Parsa and Alireza Khalilian,” On the Optimization Approach towards Test Suite Minimization”, International Journal of Software Engineering and Its Applications, Vol. 4, No. 1, January 2010.
[28]J. von Ronne, "Test Suite Minimization: An Empirical Investigation," university honors college thesis, Oregon State Univ., June 1999.
[29]T. Y. Chen and M. F. Lau, “Dividing strategies for the optimization of a test suite”, Information Processing Letters, vol. 60(3), pp.135-141, March 1996.
[30]Xue-ying MA, Zhen-feng He, Bin-kui Sheng, Cheng-qing Ye, "A genetic algorithm for test-suite reduction," in Systems, Man and Cybernetics, 2005 IEEE International Conference on , vol.1, no., pp.133-139 Vol. 1, 10-12 Oct. 2005
[31]Yi-kun ZHANG, Ji -ceng LIU, Ying-an CUI, Xinhong EI, Ming-hui ZHANG, An Improved Quantum Genetic Algorithm for Test Suite Reduction, IEEE International Conference on Computer Science and Automation Engineering (CSAE), 2011.