Saturday, November 1, 2008

Destructive vs Non-destructive testing

Do you have a destructive mind?
Now and then I have discussions about the purpose of software testing. And mainly I got the feedback that testing is all about finding defects. It is not just finding defects on a controlled way. It is finding defects in every corner. Without a found issue you didn't test well enough.
It is more the game if you have a mind that enables you to find those defects which "destructs" the application and/or process. The conclusion of this way of testing is: If a defects is found you cannot use the application.

Proof that it works
Another way of testing can be a part to proof that the functionality works. In this approach you are not focused on finding defects, the focus lies on proofing that in that part of functionality using it under those conditions no defects are found. The conclusion of this approach of testing: If no defects are found you can use the application under those conditions.

Which one is wrong?
The definition of testing according to the glossary of ISTQB of testing is:

Testing: The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.

Based on this definition there is room for both:
1. demonstrate fit for purpose: proof it works
2. detect defects: find those defects

In my opinion demonstrate the fit for purpose can only be done by non-destructive testing. Using a destructive approach you more likely proofing it doesn't fit. To detect defects can be done by both approaches.

Nondestructive testing vs destructive testing
In Wikipedia these definitions are also mentioned. It is not really focused on software testing.
Nondestructive testing
Destructive testing

In the last explanation conditions are given when to use a destructive testing approach:
Destructive testing is most suitable, and economic, for objects which will be mass produced, as the cost of destroying a small number of specimens is negligible. It is usually not economic to do destructive testing where only one or very few items are to be produced (for example, in the case of a building).

Is it economical to destruct your application?
It depends.
In the above mentioned definition you have to decide if there are more pieces or just a few and if it is economically allowed.
Example: if you are buying a car with the purpose to drive from A to B. Are you a driver who tries to drive 120 km/hour using only the first gear? As you already know that this will stress the engine. Only is that car build to drive in this way? I don't think so. You try to use the car as you want to use it and check if it behaves as expected.

Will a destructive approach help you buying the car? At least after this approach no-one will buy the car for that price.

Example: If you want to buy a tent one of the purposes is that it gives a shelter in certain conditions. You don't bring a bucket of water with you to proof it is water resistant. You trust that it is as this is one of the main functions of a tent. What you can do with your means is pulling the frame to simulate a "storm". If it stands you will be happy. If not you continue searching for another tent.

Will a non-destructive approach help you here? Yes: as you can check for space, color and usability.
Will a destructive approach help you here: Yes, only you are using controlled force based on expectations you might describe to that tent.

In software development, we often don't have more pieces, though we are able to create more pieces, or to correct the damage. Only this cost extra money.

Should we avoid a destructive approach of testing?
As destructions cost money we should decide in front if we have the budget to recover the damage. And in this case it is not only recover it during development (project); also to recover while we are already in production.

When used good and wise a non-destructive approach already provided enough information that the chance of errors in production is limited when using properly or as defined. Therefore a destructive approach is not really necessary. Normally in test processes the time is limited and therefore the chance that everything is tested as defined is minimized. This might be the main reason also to use a destructive approach.

If destructive is done using certain amount of force we should use it controlled, or not at all.

Approaches
As the goal and perception of testing differs you might explicitly mention your approach in the test plan. Based on this approach you can define you strategy. Here some approaches:
1. testing using a non-destructive approach
2. testing using a destructive approach
3. testing mainly focused on non-destructive testing, parallel a destructive test focus
4. testing using a non-destructive approach, and when a certain coverage is not reached switch to destructive to proof although the current positive results the system is not yet mature
5. test using a destructive approach and when a certain maturity of the system is reached, start using a non-destructive approach.

Based on the chosen approach you can decide which techniques fitting best and which metrics you are steering on.

Which one to use?
As for each approach there are arguments to use. I think it is important to inform the organization which approach you want to use based on arguments. Not because you like that approach.

No comments:

Post a Comment