Tuesday, November 17, 2009

When execution overlaps phases and types

Lately I had some experience within a project were the test execution overlapping several test phases and test types.

From a first thought I would say that it is against every principle of testing. Start a new test type while the previous is not ended yet. Actually, you didn't meet the entry criteria of that later phase.

For example: Your still busy with the integration test phase and you already started the user acceptance test phase.

It seems bad as all over in lots of literature it is written that you will encounter some problems like managing the stages, not having a clear exit point for a phase/test type. Ownership might shift with unclear borders.

I can imagine there are a lot of disadvantages when you continue with a planned test type instead of finishing the one you're testing in.
- Different people responsible for certain type, not clear who has to pay for delay
- Same people are testing in different phases, hard to monitor and direct the progress
- Failures in next phases would be prevented by earlier stages.
- System is not yet mature enough to continue testing; issues might lead to unnecessary discussions
- Different focus, discussion between functional readiness and business readiness. Some errors for business might be caused by other issues which at the end shouldn't be an error
- Question who to blame when project fails becomes more important instead how to help the business with a accepted product
- Technical issues might rise, is the infrastructure ready? is support available etc.

There are more items to think about.

Why I experienced within the project that we were able to deliver a product. The different types of testing was just supporting to provide a framework. It helped to explain to the business what we intended to do. It helped to address certain risks at an early stage which might endanger go-live. It also help to schedule the needed resources and make decisions about priorities. Some functionality was more important to focus on then others. We made the deadline happen and played with the framework we had. I believe the success was not because of the chosen method, not because of the defined phases and test types. it was because we spoke in the same language, we had the same goal and using the high level schedule of test types as a guide line, playing ground. Decisions were made together.

The main lesson learned here is that a method provide a framework and some boundaries. Those boundaries must be crossed to obtain optimal result. Focus should not be on maintaining the method; the focus must lay on supporting the organization.