Mutation testing is based upon seeding the implementation with a fault
(mutating it), by applying a mutation operator,
and determining whether testing identifies this fault.
A mutated program is called a mutant and if a test case distinguishes
between the mutant and the original program it is said to kill the mutant.
The idea behind mutation testing is quite simple:
given an appropriate set of mutation operators,
if a test set kills all of the mutants generated by these operators then,
since it is able to find these small differences,
it is likely to be good at finding real faults.
Mutation testing may be used to judge the effectiveness of a test suite:
it should kill all the mutants.
Similarly, test generation may be based on mutation testing:
tests are generated to kill the mutants.
Interestingly, many test criteria may be represented using mutation testing by simply
choosing appropriate mutation operators.
While mutation testing is a powerful and general technique,
a number of associated problems have limited its practical impact.
One of these problems is that the standard set of mutation
operators may lead to a vast number of mutants.
However, it has been noted that this problem can be reduced by an
appropriate choice of mutation operators.
The following are the other main problems that we have
Semantic mutation testing: rather than mutate the program or model,
which is a syntactic entity, we can mutate the semantics of the
language in which it is written.
The aim here is to capture a different class of possible mistake:
one that results from a misunderstanding of the semantics of the
language used rather than from a syntactic slip.
While a mutant is syntactically different from the initial program it may
have the same behaviour as the program: a mutant may be
equivalent to the original program.
Clearly equivalent mutants can never be killed and thus it is important,
but often difficult, to identify them.
Some approaches to detecting equivalent mutants and reducing
the number of equivalent mutants produced have been introduced.
Some mutants, called stubborn mutants,
may be non-equivalent but difficult to kill.
- Extending mutation testing to state-machines that have probabilities
and/or stochastic time.
The following papers have come out of our work on semantic mutation testing;
they are either about semantic mutation testing itself or about potential semantic mistakes.
H. Dan, R. M. Hierons, and S. Counsell:
A framework for pathologies of message sequence charts,
Information and Software Technology (to appear).
H. Dan and R. M. Hierons:
Controllability Problems in MSC Tests,
The Computer Journal (to appear).
J. A. Clark, H. Dan, and R. M. Hierons: Semantic Mutation Testing,
Science of Computer Programming (to appear).
H. Dan and R. M. Hierons: Semantic Mutation Analysis of Floating-point Comparison,
5th IEEE International Conference on Software Testing, Verification and Validation (ICST 2012), 2012.
J. Clark, H. Dan, and R. M. Hierons, Semantic Mutation Testing, Mutation 2010.
Haitao Dan, R. M. Hierons, and S. Counsell: Non-local Choice and Implied Scenarios, IEEE International Conference on Software Engineering and Formal Methods (SEFM 2010), 2010.
The following are also related to mutation testing.
M. S. AbouTrab, S. Counsell, and R. M. Hierons: Specification Mutation Analysis for Validating Timed Testing Approaches Based on Timed Automata, 36th IEEE Computer Software and Applications Conference (COMPSAC 2012).
S. Segura, R. M. Hierons, D. Benavides, and A. Ruiz-Cortesa:
Mutation Testing on an Object-Oriented Framework: An Experience Report,
Information and Software Technology, 53 10, pp. 1124-1136, 2011.
R. M. Hierons and M. G. Merayo:
Mutation Testing from Probabilistic and Stochastic Finite State Machines,
Journal of Systems and Software, 82 11, pp. 1804-1818, 2009.
R. M. Hierons and M. G. Merayo:
Mutation Testing from Probabilistic Finite State Machines,
Third Workshop on Mutation Analysis (Mutation 2007),
September 10-11, 2007.
- K. Adamopoulos, M. Harman, R. M. Hierons:
Mutation Testing Using Genetic Algorithms: A Co-evolution Approach,
AAAI Genetic and Evolutionary Computation Conference 2004 (GECCO 2004), pp. 1338-1349, 2004.
M. Harman, R. M. Hierons and S. Danicic:
The relationship between program dependence and
San Jose, California, USA,
pp. 15-23, 2000.
R.M. Hierons, M. Harman and S. Danicic:
Using Program Slicing to Assist in the Detection of Equivalent Mutants,
The Journal of Software Testing, Verification, and Reliability,
9 4, pp. 233-262, 1999.
Back to Rob Hierons'
Last updated: May 2010.
Disclaimer The contents of this page falls
outside the responsibility of Brunel University.