After reading another article how to measure more and earlier and by doing that saving money at the end. It seems so obvious that people are right when we establish a review process before we start testing. People are claiming that issue found early in a development phase it is also cheaper to solve instead when it is on a productive system.
I have heard and read that this is a written truth. I won’t object to that statement. What I’m worrying about is that people are using at any time the argument: “it is cheaper to establish a review process, or test earlier so issues are found early and therefore it will be cheaper.”
In my opinion, you have to be careful claiming this statement. You first have to earn money before you can calculate the benefits of finding issues earlier. To do this a system must be in use. You cannot spend what you didn’t earn; you cannot count in terms of cheaper when it will delay productivity and therefore cashing the earnings is also starting later on.
My advice is:
1. When you start a project: also define budget for reviewing/ testing earlier
2. When you start a project and budget is less: make wise decision based perhaps on expected value and risks where focus should lie on: sometimes reviewing/inspections/ testing earlier might be cheaper and beneficial above extensive executing lost of test cases.
3. When project is already started and budget is unlimited: you might consider reviewing/inspections of code, test cases, testing earlier etc.
4. When project is already started and budget is fixed: be careful demanding and initiating activities which cost extra money and time, they delay is not covered in the project plan and earnings might become delayed
5. Consider: When project is ended: check if it was necessary to review and if review would allow you delay of deliverance to production
6. Consider: When system is delivered, check if value is delivered and expected money is earned. Answer also the question: would adding a review process make the earnings higher?
7. Consider: If earnings would be higher due to minimizing the costs, would minimizing also result in lesser time to production?
There are enough arguments to start reviewing, inspections or testing earlier, keep in mind that you first have to add value to the business by the application, and then try to add more value if it cannot be done immediately. Sometimes this means that you have to go to production with known bugs to make money to find and solve more bugs.
Testing The REAL Requirements
2 days ago
There are several things popping up in my head while reading through your entry. They all sum up to a quote I read in Weinberg's QSM series. Here it is.
ReplyDeleteThe Candidate Product Rule:
Actually, it ain’t nothin’ ’til it’s reviewed. (Quality Software Management Vol. 2 - First-order measurement, Gerald M. Weinberg, Dorset House, page 290)
Based on the experiences I made in the last few years, I would claim
"Actually, it ain't nothin' 'til it's tested."
But this statement of course is based on the context. Skipping testing or skipping reviewing is a vicious cycle, and Weinberg shows this in Volume 3 of his QSM series. The short-term relief my benefit your project, but the long-term problems you introduce by doing so, are not worth much. Always remember: skipping testing only increases the illusions about your software, and always makes problems much more worse.
Hello Markus, you already convinced me reading the QSM series after reading your blog. Thanks for that. I hope in the next few weeks I can receive and learn from them.
ReplyDeleteWith my blog there is no intention to skip testing or reviewing. Only use them wise and when applicable. There are situations that accepting risks is necessary to ensure money is earned by delivering value to an organization. This can be done by deliverance with errors. In my opinion this is not bad if the decision is made wise.
This means to make sure you got information how to make the unknown unknowns moving into a direction as known unknowns. Like Rumsfeld mentioned and I wrote about in
http://testconsultant.blogspot.com/2009/02/classification-of-unknown-unknowns.html
Don’t see it as skipping testing, you skip certain tests which were not initially scheduled and only proposed while project is running and some person heard some noise, skip perhaps unit tests. If these are not planned and you are forcing them to happen, initially there was no budget. You might find errors which are cheaper to solve during the unit test period. If this is not calculated in front, then the budget must come from other places. It even might result in delay of deliverance as more errors found need more investigation. This is as always a risk of doing nothing, you might have not enough information.
What I tried to express with this posting is not always do what is written, think wise and also investigate what the impact is. Sometimes it is better to do nothing although arguments against it are available. Deliverance of the initial value is sometimes better then deliver at a later moment more value. Late deliverance also introducing risks. It is more likely like agile projects, deliver shippable products instead of whole product. Perform only the tests which are needed at that moment.
A vicious circle might have also his strengths, at a certain moment you are back at the beginning. It depends if you see this as a weakness or strength :)