International Journal of Education and Management Engineering (IJEME)
IJEME Vol. 10, No. 5, 8 Oct. 2020
Cover page and Table of Contents: PDF (size: 223KB)
Experimental Comparison of Mutation Testing Tools for C Sharp Language
Full Text (PDF, 223KB), PP.28-34
Views: 0 Downloads: 0
Author(s)
Tariful Alam
1,*
A. G. M. Zaman
1
Index Terms
Software Testing, Mutant, Mutation Testing, Mutation Score, Mutation Testing Tools, Test Case, C Sharp Language.
Abstract
Mutation testing is a popular software testing technique, that inject artificial faults into the program and requires test cases to reveal these faults. In this paper, an experimental comparison is analyzed for different types of mutation testing tools in C Sharp language in .NET framework. Different mutation testing tools are giving different mutation score for a program. The objective of this paper is to investigate why the mutation score is different for different tools, and the scope of generating mutants depending on different types of operators. Four tools, such as, Visual Mutator, Cream, Ninja Turtles, and Nester are selected and applied to a program and analyze the outcome. Among these four mutation testing tools, Visual Mutator is better because of its higher mutation score, and it generates mutants for both common and uncommon standard operators and object level operators.
Cite This Paper
Tariful Alam, A. G. M. Zaman. " Experimental Comparison of Mutation Testing Tools for C Sharp Language ", International Journal of Education and Management Engineering (IJEME), Vol.10, No.5, pp.28-34, 2020. DOI: 10.5815/ijeme.2020.05.04
Reference
[1] Jia, Yue, and Mark Harman. "An analysis and survey of the development of mutation testing." IEEE transactions on software engineering 37.5 (2011): 649-678.
[2] Papadakis, Mike, Nicos Malevris, and Marinos Kintis. "Mutation Testing Strategies-A Collateral Approach." ICSOFT (2). 2010.
[3] Zhang, Lingming, et al. "Regression mutation testing." Proceedings of the 2012 International Symposium on Software Testing and Analysis. ACM, 2012.
[4] Guru99. (2009, 20 jan). “Learn Mutation testing” Retrieved from http://www.guru99.com/mutation-testing.html.
[5] Rani, Shweta, Bharti Suri, and Sunil Kumar Khatri. "Experimental comparison of automated mutation testing tools for java." Reliability, Infocom Technologies and Optimization (ICRITO)(Trends and Future Directions), 2015 4th International Conference on. IEEE, 2015.
[6] R. Lipton, “Fault Diagnosis of Computer Programs,” Student Report, Carnegie Mellon University, 1971.
[7] Jeremy Jarrell. (2010, 17 June). Retrieved from www.simple-talk.com/dotnet/.net-tools/mutation-testing/
[8] Filip Van Laenen (2012, April). Retrieved from www.accu.org/index.php/journals/1929
[9] Pawar Sujata G. and Idate Sonali R., “Mutation Testing Using Mutation Operators for C# Programs” International Journal of Advance Research In Computer Science and Software Engineering, Research Paper Vol.4.7, July 2014.
[10] J. Offutt and R. H. Untch, “Mutation 2000: Uniting the Orthogonal,” In Proceedings of the 1st Workshop on Mutation Analysis (MUTA-TION’00) , published in book form, as Mutation Testing for the New Century. San Jose, California, 6-7 October 2001, pp. 34–44
[11] T. A. Budd and D. Angluin, “Two Notions of Correctness and Their Relation to Testing,” Acta Informatica, vol. 18, no. 1, pp. 31–45, March 1982.
[12] A. J. Offutt and J. Pan, “Automatically Detecting Equivalent Mutants and Infeasible Paths,” Software Testing, Verification and Reliability, vol. 7, no. 3, pp. 165–192, September 1997.
[13] J. Offut and N Li. (2011) µJava Homepage https://cs.gmu.edu/~offutt/mujava/.
[14] B. H. Smith and L. Williams, “An Empirical Evaluation of the Mujava Mutation Operator” in Testing Academic and Industrial Conference Practice and Research Techniques-MUTATION, 07, 2007, pp. 193-202.
[15] I. Moore, "Jester-a Junit test tester," in 2nd International Conference on Extreme Programming and Flexible Processsed in Software Engineering, Italy, 2001.
[16] L. Madeyski and N. Radyk, "Judy a mutation testing tool for Java," IET Software, vol. 4, no. 1, pp. 32-42, 2010.
[17] GitHub Retrieved from https://github.com/Halleck45/MutaTesting.
[18] GitHub Retrieved from https://github.com/padraic/mutagenesis.
[19] VisualMutator Website,https://visualmutator.github.io/web/.
[20] CREAM Website,www.galera.ii.pw.edu.pl/~adr/CREAM
[21] Derezińska, Anna, and Anna Szustek. "CREAM- A System for Object-oriented Mutation of C# Programs." Annals Gdansk University of Technology Faculty of ETI 5 (2007): 389-406.
[22] Ninja Turtles Website https://ninjaturtles.codeplex.com/.
[23] Nester Website, www.nester.sourceforge.net/.
[24] Nunit Website, www.nunit.org/.
[25] NCover Website https://www.ncover.com/
[26] Microsoft website: https://www.microsoft.com/en-us/
[27] R. A. DeMillo, R. J. Lipton, and F. G. Sayward, “Hints on Test Data Selection: Help for the Practicing Programmer,” Computer, vol. 11, no. 4, pp. 34–41, April 1978.
[28] R. G. Hamlet, “Testing Programs with the Aid of a Compiler,” IEEE Transactions on Software Engineering, vol. 3, no. 4, pp. 279–290, July 1977.
[29] Dong Jun,Xu Miao,Pan Yun-he,Ye. Statistic Model-Based Simulation on Calligraphy Creation [J]. Chinese Journal of Computers, 2008, 31(7): 7720-7725