There are many different approaches to testing, including alternative test criteria and a variety of techniques for finding test suites that satisfy a given test criterion. So, how can we choose a test criterion and/or techniques for a given problem? We need ways of comparing alternative approaches.
Hamlet observed that ideally we would like a comparitor that says something such as: test criterion C1 is better than test criterion C2 if whenever a test suite satisfying C2 finds a fault then every test suite that satisfies C1 will find the fault. However, he also showed that real test criteria are not comparable in this way. Thus, people use weaker relations between test criteria such as subsumes.
While we cannot make strong general statements about test effectiveness, this is not the problem the tester is typically intested in solving. Instead, the tester wants to know what is 'best' for their system. It is quite possible that strong comparisons can be made for a given system or, more usefully, for a class of systems or a class of potential faults. Such comparisons might allow us to make strong useful statements about test effectiveness by sacrificing some generality.
Back to Rob Hierons' home page
Last updated: 11th August 2004.