Somehow the question about how many test cases is so important for "important" people. As if they are getting paid by numbers instead of valued for delivered value. Somehow a false trust is derived from figures. People rely on numbers and assume that number of test cases represents good quality. This seems so obvious the way of doing.
The posting of James Christie triggered me to answer using my weblog.
I value the blog posting of But how many test cases? written by James Christie by its content in relation with his defined context. This context is lost when you translate it to numbers. Below you will find several attempt to make a good posting valued wrong.
Imagine that your blog posting is rated by the number of letters. In your posting you use about 5534 characters. Telling the same story using twitter you need over 40 tweets. Does this show value? It seems that one post in a blog provide more value then one tweet, although the tweet which pointed me to that blog was also very valuable. So is 1 more then 40?
Or what about the coverage of letters. You used all letters of the alphabet, this means your coverage is 100%, does this provide some information about the quality of your posting?What about assigning numbers to it?
Impressive usage of the letter “e” based on numbers it is far most the extensive used letter. Does this provide information? I don’t think so, Perhaps the letter “e” should be used more often, perhaps in relation with other letters. Even this way of thinking is wrong. It doesn’t tell any thing about the context.
What about visualizing the numbers. Below you see a snap shot I you look only at the numbers mentioned in his blog.
I also left some noise in it. The (con)text is now removed.
What does it value now? What information can be obtained? Perhaps the 100.000 mentioned in the text is impressive
Is it correct to drive our testing on numbers? Is it useful to explain coverage in terms of test cases executed? Is the weblog of James in the based on my examples valid and good? The numbers are clear and proven? Are you counting the time?
I compliment James with his blog. A lot of time is spend to "proof" something which is explained incorrect. Wrong questions are asked.