It is not always what it seems

While reading this book from Oliver Sacks: "The man who mistook his wife for a hat: and other clinical tales" I was triggered by the chapters who are dealing with losses. In those chapters stories are told about people who have to feeling they are loosing some parts of their mind or body. or they even don't know they are missing capabilities.

What I learned from those stories was that obvious simple identified losses can lead to different judgements. It is obvious to say when a person is not able to use an arm the arm is not functioning. When looking more in detail you can learn more from it. There might be other symptoms who can help to create the proper judgement.

I noticed in software testing we also make judgements. Sometimes under time pressure wrong judgements. I can imagine that those conclusions are drawn based on experience and learned methods/tricks. This is a pitfall we all live in. We should be aware when making judgements and exposing them to others that we question ourselves if the conclusion is based on the symptoms, symptoms combined with experience or s our experience leading.

There can be more reasons why things are failing. There might be other directions we have to look at. I think it is human nature combined with experience that we are tending to look at complex situations. In the mentioned book a quote was used to remember us that we are missing the obvious things.

In chapter three Oliver Sacks quotes the philosopher Ludwig Wittgenstein: "The aspects of things that are most important for us are hidden because of their simplicity and familiarity. (One is unable to notice something - because it is always before one's eyes.) The real foundations of his enquiry do not strike a man at all. Unless that fact has at some time struck him. - And this means: we fail to be struck by what, once seen, it most striking and most powerful. "

While reading this book I noticed that things can be different then they are. We can learn and should learn also from other disciplines. I would recommend this book to all other testers not to become a neuropsychology specialist, but to reshape your vision. It might help you when you make judgements to rethink them before you expose them. I believe that looking at other disciplines and combining/ translating them to software testing can help you (re-)define you borders of testing.

Is finding defects earlier cheaper?

Most of us are aware of the Boehm-curve which explains that finding defects in an earlier stage is cheaper to solve.
I'm sure there are lots of arguments available to stick to this statement. Unfortunately I noticed that a lot of testers are using this argument to test almost everything. They are extending the test phases. They are giving advice not to ship the software to productive systems as there are lots of risks.

There might be a slight difference between newly build software and software under maintenance. The major thing is that software becomes bad not because the number of issues which are in the code, it becomes bad because the functions, framework which comes with the code is not providing enough service to make the business using it.

Even with bad software people can work. As long as the benefits of earning money with that code is more then it would cost to solve issues immediately it might be worth to ship and use the software.

For example: to withhold software to use, you are not able to gain money which supports the business case. If you delay the system with a few months, are you not loosing you competitive strength or loosing the money you intended to make? If you are able to identify the strong and the weak points in the software, if you are able to explain what and how to use it and you are still able to make the money based on the services you are able to deliver now, why not ship it?

Finding issues might be cheaper to solve in earlier stages, only you need first the ability to make money before you can extend the project to find and solve everything. I can imagine that is the system is mandatory to continue the business you are also willing to accept the risk of having issues in the system. It is more important to be able to act promptly when issues becomes problems.

I would rather suggest instead of claiming it is cheaper to find issues earlier and therefore continue testing, be honest and ask yourselves if you are able to provide information on which clients can make there decisions. Don't fall in the pitfall to claim that this can only be told based on coverage. If you are able to explain what you have done and how you see the system and what you didn't do and you have faith in you and your team. You already made a huger step then speaking about coverage. If you are able to tell what functions might possible cause risk and how you are tending to monitor it and you are able to help fixing it within time you are more capable then telling you tested well and give negative advice as there are issues which might bother some people.

It might depend on the development method you choose and the organization who stands behind you if this can be successful. I noticed situations were it was cheaper to ship products with some issues or some risk. Money was made immediately instead a few months later and the issues which were found in the first days were faster solved and tested then during the normal test project.

So it finding defect earlier cheaper to solve? Yes, is it cheaper to withhold the system to go production? I think most of the times we have to rethink the verdicts before making our statements.

When execution overlaps phases and types

Lately I had some experience within a project were the test execution overlapping several test phases and test types.

From a first thought I would say that it is against every principle of testing. Start a new test type while the previous is not ended yet. Actually, you didn't meet the entry criteria of that later phase.

For example: Your still busy with the integration test phase and you already started the user acceptance test phase.

It seems bad as all over in lots of literature it is written that you will encounter some problems like managing the stages, not having a clear exit point for a phase/test type. Ownership might shift with unclear borders.

I can imagine there are a lot of disadvantages when you continue with a planned test type instead of finishing the one you're testing in.
- Different people responsible for certain type, not clear who has to pay for delay
- Same people are testing in different phases, hard to monitor and direct the progress
- Failures in next phases would be prevented by earlier stages.
- System is not yet mature enough to continue testing; issues might lead to unnecessary discussions
- Different focus, discussion between functional readiness and business readiness. Some errors for business might be caused by other issues which at the end shouldn't be an error
- Question who to blame when project fails becomes more important instead how to help the business with a accepted product
- Technical issues might rise, is the infrastructure ready? is support available etc.

There are more items to think about.

Why I experienced within the project that we were able to deliver a product. The different types of testing was just supporting to provide a framework. It helped to explain to the business what we intended to do. It helped to address certain risks at an early stage which might endanger go-live. It also help to schedule the needed resources and make decisions about priorities. Some functionality was more important to focus on then others. We made the deadline happen and played with the framework we had. I believe the success was not because of the chosen method, not because of the defined phases and test types. it was because we spoke in the same language, we had the same goal and using the high level schedule of test types as a guide line, playing ground. Decisions were made together.

The main lesson learned here is that a method provide a framework and some boundaries. Those boundaries must be crossed to obtain optimal result. Focus should not be on maintaining the method; the focus must lay on supporting the organization.

Growing bananas

This week I attended a presentation Exploratory Test Automation:Investment Modeling as an Example given by Cem Kaner in the Netherlands as part of the Cem Kaner week

One of the statements he made is avoid commodity. This triggered me to reflect on how I see software testing.

To my opinion Cem Kaner tried to explain to use why commodity is good for those who feel happy to be a tester who have and can use standardized skills and knowledge. Who accepting to be replaced easily and who’s expertise can be cheaply outsourced. Those testers are as valuable for the client as all other testers. The other side is that there are testers who are willing to become better; they want to become valuable for the client. The client experiences the value. I think for this you have to express yourselves, keep learning, and adapting to the environment and work hard for it.

What does this have to do with growing bananas?
As C. Kaner mentioned about commodities:
There are green bananas
and ripe bananas
and rotten bananas
and big bananas
and little bananas.
But by and large,
a banana is a banana.


If you look at the phrase above and replace "bananas" with "testers" then testers are in general commodity testers. Unless they choose not to be a commodity tester. If the choice is made then you have to become aware of yourselves, what do you value?

As with a banana, the shell can be used, only the inside is eaten. Based on that taste the banana is valued.

This made me think a bit further. In general if people are thinking about bananas they visualize the banana including the shell and sometimes with a small piece of the inner stuff visible. People are visualizing a banana eatable when the shell is removed. There is more to imagine. the structure of the inside of the banana is also a result of the outside structure, how it is grown, how fast, with which means and under which conditions.

The same can be the situation for testers, they might be thought under certain conditions. This might not definitely lead to wrong results. They remain testers. I can imagine that for certain test certification might be useful to gain some basic fundamentals about testing. It might be right, it might be wrong. At least a direction can be defined. It is up to the tester if he/she becomes a commodity tester or able to add other specific value to the client. There are more roads which leads to Rome.

If testers are tending to become commodity testers and they might be identified as bananas. As said, bananas shouldn't be judged and testers also not; just because their believe in methods and approaches is different. I think it counts more what a tester is doing to become of more specific value.

Serving the method

In a discussion with a fellow tester we talked about testing effort. The situation was deliverance of certain request for changes. It contained functionality of which the persons who wrote the functional designs claimed it hadn't that much impact. The users who were involved also claimed it can be tested intensively spending several days or just the main things accepting risks.

There were also users involved which had their own history with the system. They wanted to test the system based on their experience from the past.
The fellow tester admits there might be more options to test which the others didn’t think about. He based that idea on the method he has been thought. From theory, he might be right. He also used the arguments based on the method. One of the main arguments was: the quality must be right and optimal. To claim this we should spend a lot of test effort.

In the discussion it was not about who has right and who is wrong it was about the effort needed and if it was valid to use a test method as argument for the direction.

When people are discussing, there are several views. As the fellow tester doesn’t blog I can only present my view.

In my opinion the following points were important
- No discussion was needed as the changes were important, it was forced by top management;
- End date was fixed due to management decision;
- Resources for test execution was available because of importance;
- Resources and quality of documentation to derive test cases using test techniques was not sufficient available;
- System was available;
- Support from development and business was sufficient;
- Solutions would be delivered in time and on time, just before the deadline;
- Time was left to follow phases from TMap, no time left to identify and cover all risks;
- Knowledge of application managers and business was more important to select test cases then the documentation and test techniques;
- Cost of not delivering was larger the solving after implementing on production.

When looking to a method like TMap, it wouldn't be possible to implement a solution on productive system as not enough proven information was provided about the risks. No defined and measured coverage was made visible. Requirements were too poor. No test techniques were selected and used which were mentioned in the method. Still we made it to deliver successfully to productive system.

Maintain testing knowledge

I recently stopped smoking. It is about 4 weeks ago I didn't smoked any cigarette. During these weeks I noticed my focus switched to other important things besides testing, work and sitting behind the computer writing my blog or participating to forums related to testing. Somehow I needed all the attention to focus what was important at that moment: continue stop smoking.

The worse part of these weeks was the emotions which were involved. I can tell you, those were hard. Sometimes they are still hard. I stopped cold-turkey, which means, no means, just attitude and immediately.

Of course, there are courses which help you stop smoking. there are several methods which help might help. There are tools and medicines which might help stopping. I didn't try them before as I wasn't convinced they would help.

This made me wonder about testing. If in a normal process emotions are involved, why not in testing. I'm certain emotions are there, only can you show me a book related to testing which is dealing with emotions? If books are not writing about them, how can methods be successful? Emotions should not be neglected.

Another thing which slipped my mind, why spend so much time about investigating the proper method, choosing a method and implementing a method. When decision is made, it might const too much time and money to convince people why the chosen method is good and working. Perhaps just starting with testing is already sufficient and having an open mind set for what crosses your road.

I can imagine that when no borders are created by methods, creativity will grow and also ability to adapt. This will help you learn faster and provide means to get to your goal.

Another thing is learned behaviour. Some people say smoking is a bad habit. I did it for 17 years. And in those years I felt quite good about it. Only now I have to learn to live without it. What about testers. If newbie testers are learned with certifications, is this as bad as smoking? For some people it seems bad, for others not. The same with testers who learned certain behaviour in testing. they were thought to think in a certain way and became specialist in that way of thinking. Is that good or bad? Will they have a similar route to go when to stop thinking that way?
Which tools, methods, people are available to help, guide those?

I wonder which people are able to tell they are right and others are wrong. When this happens, also emotions are involved, emotions which are not written in books. What can be learned, and how can testing skills be maintained. I noticed in these four weeks, I neglected some part of learning.

Investigating details, a waste of time?

I had a great holiday this year. With my family we lived in a tent for 2 weeks. Every day was fun and gives me the opportunity to have some quality time with my family. Of course there were some moments which good be better. It was great because the big picture fits.

When sitting in my chair in front of the tent I had some time of thinking. Clearing my head of thoughts I filled them with news ideas. I needed something to do and made some pictures from close distance. This gave me the idea that although the pictures are nice it doesn't tell the whole story of my holiday. This made me wonder why we are focussing on details in testing. Why we are for example insist on unit tests. one of the reasons is to find issues sooner. Should this be done as in a huge system there might too much details. For example, if I shoot every detail of my holiday I had to spend more time on defining details and making the pictures instead of enjoying the holiday. Because of this I would miss a lot of joy.

Below you find some pictures I shoot. With these examples I will try to explain why focussing on details is not always the right way.

Foot-picture:
My son asked me to make a funny picture of his foot. As you see, you can notice some details. Only is this enough, at least you can check that there are no wounds on it. Ask your self, if there was a wound and it should be nursed, what sense would it make? Is it necessary for the whole picture? The grass behind the foot gives a indication that there is more space behind that foot. Currently the foot is the main object. If I would fade-out it would become part of a bigger picture. In that big picture a lot of other things are happening like children playing, parents sitting, tents standing, trees are growing. As a foot is is part of that bigger picture. I would ask you: If the foot has a wound, what influence would that wound have on the bigger picture? Should it be nursed? Was the time spend on this detail valuable for this bigger picture?




The grass-picture:
Looking behind the foot you see grass growing. When you are on a camping site you have lots of grass growing. When I would make pictures of occurring circumstances which it would certainly result in pictures with some pieces of grass. Is it necessary to investigate every time the details of it? With the picture below I could ask several questions about the colour, the structure. How is the single grass halm growing? Are there different types of grass, is the floor covered well? Are there some dangerous insects living over there? Are there laying pieces of glass lying there which might hurt the children? I wonder is it necessary to ask these questions every time to value the bigger picture? Should I spend in testing all the time effort to similar objects only on different places? Should all details be covered?


The wooden stick-pictures:
Even when there is a need for investigating details, is it done right? In the example below there would be a need to investigate the wooden stick. As you see, the wooden stick can be tested in similar ways on both pictures. Only there is a slight difference between the pictures. So when do you know when you are using the correct view? How can you tell from which angle you should test? I you focus only on the stick, what can you tell about the size? Perhaps it is not a stick, instead a tree. Due to the size it might hurt and have impact on my holiday. As you see, to check the size, you have to focus on other details than the stick. You have to look at the position of the stick with respect to its size and the environment. Focusing on the details of the stick will cost too much time and wrong conclusions might be drawn.




The ash-tray picture:
Sometimes the detailed image looks very dirty you hardly can continue watching at it. For instance the code is very sloppy code. You get distracted from the bigger picture. Perhaps this is a result which supports other issues. In the ash-tray you see a lot of dirt. It is unhealthy etc. Still it is functional. The ash is stored in some kind of a container and can do less harm when there is no wind. If the ash-tray was not there, the output of other habits would be stored else were in an uncontrolled environment. Is it necessary to check in detail how it looks like?



Beer-can picture:
This picture is one I like, it shows some important information from the beer-can in detail. Only I know it is a beer-can as I drank used it. You only see the top of it and can perform some test on it. How would you know if you are sufficient? When are details good enough? If these questions cannot be answered what sense does it make to perform detailed tests.



Food-picture:
Sometimes the details look like a mess; you have to focus more to see what is in it. After focussing you will see objects which can lead to further testing. Would it make sense? The picture below is from a dinner we had, although it looks awful it tasted very good. And it was just to combination which made it taste good. The taste was the main object of the food;it should fulfil one of our needs: dinner and we should be able to eat it while it is hot. For those objectives it was not necessary to have each piece lying in a specific order.




Towel-pictures:
In both pictures of the towels you see details about the structure. The first is made from a larger distance then the second one. How would you know that the detail you look at is sufficient? Wasn't it good enough to know that towels were available and trust the purpose of them, drying your hands? Looking at the towels you also see that the sun is shining. Sun was an important factor of success of our holiday. Do we need this picture the draw this conclusion or should I made another picture with the towels?

Conclusion
With this story I didn't want to tell that spending time at details is not useful or unnecessary. It should make you think if it is useful to look at details any time. Is it mandatory to perform unit tests? Of course people might claim that according the Boehm-law, found issues in an early stage is much cheaper to solve. You have to ask your selves every time, is it needed to find issues in those areas? I hope the examples given above made you think a bit about the meaning of details and focusing on details. Of course I could have made other pictures as well about the holiday and also there the question can be asked, is it detailed enough or not. It is this what should be done: keep asking this questions.

Magazine: MacroTesting

A new magazine is introduced related to software testing. It is called: Macro Testing.

It is free after subscribing to the site: http://www.macrotesting.com/

After subscription you are able to view the magazine online and also download every article separately. Based on a request mail I assume that the paper version is shipped to the company address.

Why only focussing on bugs and quality?

Reading all kind of blogs, books and articles we all keep focussing on bugs and quality. There are persons who claim that finding bugs is the main goal for testers. Other people mentioning that we have to advice on quality.

Focus
Sometimes they write about skills other then testing skills like domain knowledge, technical skills, communication skills and more. And more often these skills are claimed to be needed to find more bugs or get better metrics how to present quality.

There are also people who use the words like "added value for customer" and are trying to proof this based on number of bugs the prevent to go into productive system. The pitfall I see in these approaches is that the view of the tester is narrowed from the start. They focus on how to find bugs fast or with certain coverage; they try to convince the customer about quality of a product. While focusing, they used their best practices and lessons learned based on these principles.

This raises the question by me: Are we doing the right thing and can we do better? Obviously we do, as the customer is happy and the product went live without major problems. Is it?

Doing right?
I often hear that if we have to go from A to B we have to know were we are, were A is and how we can get to A. Then we use our knowledge to get to B. During the journey moving towards B we adapt our approach. Another thing I hear often is: Why to invent the wheel again?

For this we use methods and approaches to start from A. We start testing and find the bugs as without no bugs in other men’s view we are not doing the right thing. because we are always under time pressure we tending to adapt our route to get to B to make sure that "our" process will not be the project failure. So w are not adjusting our approach to add value, we merely adjusting our approach to hide the failure of our lessons learned/best practices. If this is the case, are we doing the right thing? Are we focussing on the items which need focus?

Multiple processes
When hiring testers often are skills asked related to: domain knowledge, branch knowledge, programming knowledge, test tools, certifications about testing, communication skills, knowledge about programming methods and sometimes testing skills (something different then testing certifications). Indirectly they might support the test process as defined. Is this still a guarantee that bugs are found, that quality is proven?

If you answer this question with "Yes": Why is it that the benefits of this knowledge are not measured? As far as I know you have to pay for every bit of extra knowledge/skills. It seems that there must be some indirect value hidden behind this knowledge for the test process.

Re-focussing
This indirect value can only be a result of involvement of other processes. These processes providing information for the test process which is valuable. It is important to know what kind of impact this information has on the test process. For example: if branch knowledge is important, then we not only should define a test process which is using this knowledge, it also has to take care of gaining, interpreting and processing information from and towards the people who are working in that company. This is other information then number of bugs or visualization of quality.

I think we also should try to make the indirect value of information from other processes move to direct value. We should also focus on processes that generate information which has impact on the test process. This might result in manager who are able to value information from those processes and add value to them with respect to bugs and quality. This might lead to a situation that a product is shipped to production with unknown bugs and poor quality in certain areas, although it has more value for the processes when shipping it know then waiting when everything is uncovered and tested.

Testing becomes more
With this idea, testing becomes more then finding bugs or providing information about quality, it also process information from other processes and value the product and skills of team members against those processes. The focus of testing becomes broader then just the test process. the test process shall also value the information with respect to the other processes. Pieces will fall together.

10 Lessons when waiting for a fix

Our dishwasher is broken. It bleeps and light flashes quickly for four times. The water is not pumped away and after trying to reset the machine, it keeps offering water. Don’t do this at home because after a few trials the floor gets wet.

As always, when something breaks the time is never right. With dishwashers, they break when you need them, or not? In this case we noticed it wasn’t working because there were some dishes to clean. When the machine stops, it really stops. To process for a solution might be quite interesting.

When the machine told us there is something wrong I went to the machine and investigate what symptoms there are:
- Water in the machine
- Clean dishes (it stopped at the end of a cycle)
- 4 Short bleeps and blinking light
- Reset button functions
- Restart shows same behaviour after a few minutes
- Water was offered to machine
- No other noises

With this information I went to the internet and performed a search based on the symptoms and brand of the machine. Somehow I didn’t had the information available about the specific type. I found several threads mentioning several options to do:
1. Clean the machine using some special cleaning stuff
2. Hire a mechanic
3. Do it yourself (some “detailed” information how to do this was also offered in terms like: remove screws, check, be careful)

As it was weekend when this behaviour initially started I couldn’t do anything at that moment. This bothered me and made me feel bad since we got used to this delightful machine.

As a good husband I tried to fix it myself the easiest way, telling my wife to clean the machine by buying the tablets which should do the trick. As the machine was still under guarantee she called the store and heard that if the machine is the problem, only the call out charges has to be paid. If the problem is blockage of the filter, then we had to pay also the hourly fee.

After a short discussion with me by wife bought and used the tablets and it worked for a while. Only now it is broken again. Again we tried to clean the machine which didn't work. It seems that the pollution wasn’t the problem after all.
Lesson 1: If the symptoms are gone after trying a solution, this doesn’t mean that it was the proper solution, it can be coincidence.

Together we made a decision to call out for the mechanic. They were responding very quickly, within 3 days, he will come and check under the same conditions. We accepted the risk that it didn't had to do with the machine and we might have to pay also for the hourly fee. We were able to avoid this by calling in a plumber. Only this would also cost money and no solution is guaranteed.
Lesson 2: When making a decision who should help, call in the proper person based on own investigation and others.

Today is the day, the mechanic will come. The waiting is started as the store mentioned that he would here between 8:00 AM and 6:00 PM. "Fortunately" he will give a call in front just 1/2 hour before he would arrive. I have to admit, when the period is there when solutions are entering a certain time window, uncertainty is more annoying when calling in terms of hours then in days. You have to adapt your schedule based on this uncertainty.
Lesson 3: Communicate when an agreement is made on delivering solutions. Time schedules have to become more detailed when the time is right.

While writing, the mechanic called. He will be here within 20 minutes. Now the moment of truth becomes closer, what should we pay after all? And will there be a solution provided? I already prepared myself by being able what I have done to avoid discussion that it is not the machine and that we urgently need a solution for this problem.
Lesson 4: Be prepared when a solution is offered, communication on arguments become more important otherwise you might have to pay for false reasons.

Of course, we have a workaround, doing the dishes manually. Only this is not a proper solution as the dishwasher is using valuable space in the kitchen which will become useless.
Lesson 5: Workarounds should be temporarily, as there are other costs then the usage of missing functionality.

The mechanic just left. As the outcome was not yet clear in the beginning it seems to me important to assist him removing stuff which stood in the way of investigation. I could decide that he can do it himself, only I would spent valuable time from him and since the verdict was not yet made I probably had to pay for it. During the process of investigation and problem solving I was around for assisting and answering questions, trying to avoid not being in the way.
Lesson 6: Provide support which is in your area, avoid discussions and certainly those who area out of your field of expertise.

Within minutes he found the problem. He solved it and also made some adjustments to the "infrastructure" as he couldn't find the cause of it, only noticed that there was something wrong which could be easily fixed which might avoid similar problems in the future.
Lesson 7: If the problem is solved, also take some time for the situation.

After everything was set in the proper place, like the front desk of the dishwasher etc, he performed some small test. he turned on the machine and observed the behaviour. He also listened if it sound right and if the problem actually was gone.
Lesson 8: Let the "problem-shooter" convince himself that the problem is gone. This can be done by demonstration.

Now there was some time to offer him coffee. The mechanic is also human and this was a good opportunity to show respect and recognition. During this few minutes, he was a fast coffee drinker, he explained more about the situation, how it could happen and also a bit about his work.
Lesson 9: Show respect and recognition in a proper way. You perhaps need him again. Don't exaggerate.

At the end, the bill has still to be paid. Fortunately, we had to pay only for the call out charges. The mechanic explained that it felt under the guarantee conditions as he couldn't find the exact cause. He also provided information about situations it wouldn't be part of the guarantee.
Lesson 10: Ask for information about the situation so you are able in the future to narrow your initial decision about what to do.

What is the leading source?

Ever wondered which data you should rely on? Are we monitoring the test scripts or are we monitoring the issue database? Based on which information are you able to give an advise about quality?

It seems obvious to claim that both sources are important and necessary. Both sources appear on those metrics dashboard often are used. Which means that information shown there is used to provide detailed advice whether to go into production or not.

Only what are you doing when they are not in sync? This should not happen? Of course it does!

Situation A:
All test cases are executed and all issues are closed. Based on which source is advice given? Does it matter? Yes, who gives the guarantee that you found all issues? Which source gives an indication about quality? It seems that the list of pending issues provide sufficient arguments towards management to go live. In this case, the issue database gets the benefit supported by the number and type of tests executed. the issue database is the leading source.

Situation B:
All test cases are executed and some failed, for these issues are registered and still open. It seems that test scripts and issue database are in sync providing the same information.
Are you using information the issue database or the test cases? In this situation it seems that the issue database tells nothing about what you tested. The test scripts should. if al important/critical tests are performed then you might be able to give advice based on this information and identify possible risks when going live with known issues. The test scripts are the leading source.

Situation C:
All test cases are performed and passed; only there are some issues still open in the issue database. Those were found during testing only not related to any test case.
There might be situations when test cases are based on designs and they are not telling everything. During testing new issues are found which are not related to any test case. Another situation can be that issues are found which are also in production. In these cases the test scripts cannot be used as resource to provide advice as according to them, there are no issues . The issue database will be used and based on the impact of open issues an advice will be given. The issue database is the leading source.

Situation D:
All issues are closed and not all test cases are executed. Under time pressure you are not able to execute all test cases. Though, all mandatory cases are executed. Bases on information about the test cases an advice can be given. The test scripts are now the leading source.

Situation E:
All issues are closed only the test scripts are showing that there are issues pending. According to the issue database there is no risk anymore. According to the test scripts there is some risk although all cases are executed; there are issues with pending status in the scripts. An approach would be performing a recheck on the issue database. The question here is, does the issue database provide enough information to change the status in the test script? Or, are you executing that test case again? What is now the leading source? If the issue database contains test results which proofs the issue is solved and the time stamp corresponds with the time stamp in the test script, the issue database is the leading source. if insufficient information is available, the test script becomes the leading source.

Situation F:
Not all issues are closed and not all test cases are executed. This is a tricky one. It might be easy to give a negative advice as not everything is done; test cases and issues are open. What if those are not that important? What is the leading source then? I suggest that both sources can provide some information only not enough to become the leading source. Another source should be contacted. This can be a requirement list, a risk list, the manager, the business. In this case a combination of sources should be used.

Situation G:
No test cases are executed, and some issues are still open. This seems to be odd to go live with this information. What about an update of patches. Sometimes tests are performed without writing them down. In this case the only source you can rely on initially is the issue database in combination with knowledge of people. Let's say that the issue database is the leading source.

Of course there are other situations also. The intention of this post is visualize that based on the situation the source that provides information to make decisions differs. You should be aware that you can go into production based on number of test cases or number of issues with respect to their status.

Bounded by thinking in methods and processes

It is now almost a week ago I had a challenge with Matt Heusser. One of the interesting things I learned is that approaches differ on some points. Over the years I was thought to think in processes like TMap, ISTQB , ITIL, PRINCE2. A good habit of mine is to investigate the borders of those methods.

When I was at school I noticed I drove my teachers and also fellow students crazy when questioning methods. Not to proof they will not work or they are not true. Just to investigate the boundaries of methods with the purpose to know when I cross those boundaries I need to be creative.

During the challenge I forced myself to set aside what I learned and re-shape my knowledge based on the questions I got. This helped me to find some answers and missing others. Of course this is not bad. Only there was on obvious test I forgot. Knowing it is an obvious one I forced myself thinking and start using all kinds of methods approaches etc. I noticed this brought me further from the actual solution. at the end I mentioned it, only as a side note and not as main thing. you might say I was a bit blinded by methods.
I learned that obvious things are often missed.

In the projects I worked in methods were pre-defined. In the Netherlands a method like TMap is very common. It helps you to create a structure in the test process. In almost every project it becomes a main goal to work according this method as it is already sold to create structure. When other actions are needed it is first checked against the method, does it fit in. Sometimes the approach is changed; sometimes the actions are not done because it was not agreed upon and needed as the test plan is leading. This lead to actions are skipped based on "false" arguments.

This brings me back to a phrase a teacher told me once: "Methods are tools to model your situation, keep in mind that models are just a simplification of the reality"

Based on this statement, methods and defined processes as result from methods are fallible. You never can capture reality in a model. You will always miss items. Only afterwards you might notice the importance of missed items.

When obvious things are missed based on using methods and following processes. Then, thinking in methods and processes lead to obvious blindness. The question is how huge wil this obvious blindness be? I'm sure a lot of methodologists are claiming this is untrue. There is space in methods/processes to prevent blindness. Perhaps it is. Only is this for everyone?

If everything can be dealt with by a method what level of experience and skill does one need about that method to prevent the obvious blindness? And is it acceptable to demand those level of skills to anyone? Is this reality?

A new question is: Are we dealing with reality or methods?

What would you do?
1. Accept obvious blindness and deal with it later when it fits the methods/process
2. Reject methods/processes and define your own world
3. Define space in methods/processes to address time to investigate obvious blindness holes
4. Teach project members in methods to the highest detail and claim there cannot be any obvious things which are not overseen

Whisky or whiskey and testing

Some people call it whisky and some name it whiskey. The notation depends of its origin. In Ireland and the USA they use Whiskey. In Canada and Scotland they name it Whisky.
In the Netherlands we tend to use Whisky.

With this liquor you have differences already in tastes, flavors, ages, colors, experiences. I can imagine that you also have differences in believe and understanding. Most of the people can name at least three different brands of whisky and tell which is better. There are a lesser of them who actually tried that whisky.

You have these kinds of differences also within testing. People think to talk about the same when referring to testing an application only the approach is different. Like with drinking whisky, people have different understanding and believe they talk about the same drink telling it is the best whisky. It is quality. They claim this statement because of their experiences.

If this is true, then quality is just a result of the experience something gives you. This has quite some impact on testing as people tend to get experience by using specific approaches.
For example the view of testing in the USA is a bit different then in Europe (I'm generalizing a bit). In the Netherlands we tend to test according procedures, guides, phases, project plans, steering groups etc like test approaches called TMap and ISTQB. In the USA they tend to test based on technique, their understanding of technique, heuristic methods etc.
Because of these differences the experience will also differ. Therefore the perception of quality will not be equal. As result of this; testing is not equal.

The pitfall here is that we are trying to teach each other about testing, sharing ideas and learn from each other based on different levels of understanding while information might not fit the experience. Wise lessons are misunderstand and misused.

A few days ago I noticed on twitter how a fellow tester was going to enjoy a 16 yr old whisky called Lagavulin. I never heard about this brand and believe him when he claims it is a good, quality whisky. For myself; I like the 18 yr old Highland Park and in certain situations the 12yr and the 15 yr old are also good. It depends on the situation and mood I'm in. This is an example of differences in understanding quality although we both are talking about the same topic, there are differences in experience and understanding. I could try the whisky he is drinking and offer him some of mine to get a better understanding of each other, perhaps when the occasion is there. At this moment I stick to testing.

Knowing about differences in testing I try to understand more about other ways of testing. You can read about it, experience is more valuable. For this I was lucky to visit the Miagi-do of Matthew Heusser as result of my search for a Mentor, Mentor in software testing. During these visits I received a black-belt challenge which I accepted. I will not share the details of this challenge as it will not be a challenge for you any more. That challenge was though. It was a brain teaser and breaker. It was fun to do. I think the strength of the challenge was the capability of Matt to adapt the challenge based on information I gave him.

At the end I earned the brown-belt and I'm proud of it. Not because I got a brown belt, because Matt made me learn more about myself how I approach testing, what differences there are in points of views, were I rely on my own experience and what the pitfalls are when doing this. He also strengthen my idea that there are more views in testing and to understand those I need to learn more. As I knew this already and I am more proud of gaining the brown-belt instead of the black-belt. It makes the brown-belt more valuable because I'm aware of my knowledge, willingness to learn and drive for testing.

Like drinking whisky, testing is also a case of perception. They might differ based on experiences without claiming what is good or bad. For this you have to be able to communicate on proper level of understanding and able to share thoughts and ideas.

Questions for reviews

Are you also a person who is able to find holes in functional/ technical designs and other documents? When reading a paper do you sometimes have the feeling something is missing?


I was triggered by this question because recently I was able to tell people I missed something in a functional design. Not that it was a bad design, how can it be as developers were able to write a technical design based on demanded functionality. Also were testers able to test the functionality. So was it actually that bad? No, we managed to reach our goal.

Reviews are introduced to avoid errors and changes in a later stage of development. There are lots of books written to explain which type of reviews can be done and also how to do them. I think certain issues in DUR (Documents Under Review) can be avoided also on an earlier stage. In some organizations this is mainly done by using writing guidelines and templates. Only this doesn't guarantee information is missing in those documents.

Before a document is handed over for review you can also perform some initial checks. For this purpose I post here an idea I have about this, perhaps it might help you also.
Obviously a checklist would be appropriate. The disadvantage I see of using a checklist is people stop thinking. Under time pressure they even might answer every check with "YES" so the process can continue.

Instead of using checks you might use questions to gain information about the document and how you managed to create the document. The following points can be used for questioning Functional Designs. I think it also might be used for other documents as well. As often, list can never be complete and should never be used as the complete truth, the following points should be used as a guideline perhaps as a Quick Reference Card.

History
- Is this document adding new value to other written documents related to the same functionality/processes?
- What is the level of knowledge about these documents to others?
- Is a reading instruction needed for those documents?
- Are the requirements discussed with the customer how to write them down in the document?
- Did previous changes on similar functionality lead to discussions/questions/meetings and how can this be avoided in this document?

Document Structure
- Did the writer have to adapt the content to make it fit the template?
- When did this happen: in the beginning of writing the document or later on?
- Where did this happen in the document?
- Did the writer struggle during writing the document to define functionality on the proper place of the document?
- When did this happen: in the beginning of writing the document or later on?
- Where did this happen in the document?
- How did the writer managed to use the structure making the content fit?

Usage of words
- How are used examples supporting the explicitly defined situations?
- How is it made clear that used examples are related to defined situations in other documents?
- If a translation had to be made from one language to another language are the key words of requirements used in the correct context?
- In which way are variables/conditions from the business rules made explicit and was this needed as results of common knowledge or specific knowledge?
- When and were did the writer struggled finding the proper word to describe the functionality and matching the requirement?

Explanation of functionality
- How did the writer made the relation between the functionality and business rules?
- How explained the writer unusual situations of defined functionality?
- What must be done to translate the written functionality in an image like process flow, time axes, etc?
- When and where are other chapters or paragraphs needed to support the explanation of the functionality?

Purpose
- Is there enough information in the document the writer can hand this document over to a colleague and he/she is able to explain it to the developer?
- Is the writer able to explain the relation between business rules and the functionality based on the information in the document?
- What arguments can be thought of to support the size of the document with respect to the impact and value of the requirement?

As often, more questions can be asked/defined. In my opinion the questions mentioned above can be used to question your document. I tried to write them in a way you are triggered to ask more questions. I believe when you are not triggered to ask more questions you might spend more time on your document and with your colleagues talking about the functionality.

Improving seems so obvious

I already mentioned this before, as tester we tend to continuously improve our work, make our life easier and hoping to help the customer with those improvements.

In Quality Matters issue Q1-2009 I already wrote about avoiding improvements in the article called: "Be innovative! Stop improving your test process"

The basic idea of avoiding improvements is because it might hurt other processes. It might benefit your process as you get in control, only when you ask other project members to help you like better technical design, someone has to write them. Often these are the persons who are currently needed to deliver the code you need.

Recently I was in the fortunate situation a tester told me that things could be improved. An interesting discussion took place
Q: Why improvement is needed the initial answer was to optimize the process?
A: By optimization the speed of execution of certain tests will be increased
Q: Why speed should be increased as you managed everything in time?
A: We can perform more tests and also perform some regression tests as functionality was added during the acceptance phase.
Q: What is your idea about the needed improvements?
A: We could do some clustering of test cases, we can change the templates, we could communicate better, daily kick-off meetings will help, data preparation should start earlier, maintaining metrics can be optimized....

It seems he had some thought about this and obvious those thoughts seem to be valid. Are they? I told him that perhaps improvements can be done and I asked him to write those improvements down and explain them to me. I asked him also to think about what the benefits are, the disadvantages, how much it will cost and beside what value will it have. After a day I received a small document with suggested changes, very good structured. Although the document mentioned in a clear state within the context as he saw his activities and how it can be improved it was still missing something. I was written for an "ideal" world. His "ideal" world. Together we went through this document and start talking about the necessity for those improvements and also how will it value the project/business and does his project context match with the overall project context.

It was an interesting conversation as I learned more from the suffer he faced during performing the tests. I also was triggered again that providing only one solution is never enough. It remembered me the words a teacher told us: "Always provide 3 options, let the people decide. One option is no option."
At the end of the meeting we came up with the following options:
1. Do nothing
2. Do something which is not optimal
3. Do the optimal solution

This reminded me about an posting I made here in the past: July 7th 2008 I posted an article here The power of Three about the situation that there are always at least three options.

In this case it would help if three options were offered also. Only just providing more options is not sufficient. Those options should also be placed in the proper context. It should not only help you and benefit your team; it should also be valuable for the customer.

As in my article in Quality Matters I explained about internal and external improvements, this with respect to direct and indirect improvements. I believe you have to be careful with improving. If you can do better, why not start. If you ask others to do better, consider there time and resources are also valuable. If you are asking them to change or do more it will cost something on other areas. If people are spending lesser time on those areas they become worse. So it is important to place this also in your defined context. What is valuable, their current activities for the organization or you new activities?

So again, here is also the power of three involved:
1. Does it help you?
2. Does it help the project?
3. Does it help the organization?
Add for these questions the value towards the organization. I would suggest translate the benefits and disadvantages into value to the product, organization, team.

At the end of the meeting we came to the conclusion that although most of the suggestions are valid as can be, it would cost more on other levels. Does this mean that those suggestions for improvement are useless? I certainly would say no. When people are able to come up with improvements, there must be signals that something is not going smooth. These signals should not be ignored. As they are all written down it will help when improvements are done on a structured way, they can become part of a total improvement plan. Or they can be used as signals that we should think about improvements.

Swine Flu: adapt your processes or ignorance and denial?

Over the last days I was wondering why organizations are not busy adapting their development/test processes to continue when the "Swine-flu" explodes. Is this ignorance or denial? Am I the only one who is worrying about this?

Currently the prediction in the Netherlands is that this flu will exploded in the last months of this year. According to this article: (translate from Dutch using Google translation service)
Mexican flu peak may be in August ; thousands of persons will catch this flu in august per day, the English government expects in Britain a over 100.000 of patients per day.
Im sure there are more figures which are more accurate on the internet.

I also heard that in the Netherlands they expect between 20% and 30% of the people which will become ill. When this happens also the schools might be closed. Imagine the impact of this.

Currently ICT companies are dealing with the current recession with a certain number of employees on the bench. I cannot imagine that people are able to point them as victims of the flu, everyone which is on an assignment is allowed to become not ill.

Let's assume that 20% of the people are becoming ill.
Due to a general figure, this also counts for about 20% of the project members. This will reduce skills within the project team and result in delay of deliverance of functionality and also solutions.
Next; when schools are closed, and let's assume that about 30% of the families both parents are working, someone has to take care of the children, resources are again reduced. What do you think when this hits again 10% of your project members?

What happens when your project resources are decreased with about 30%? What is the impact for you? And what happens if you are also depending on other parties which supplies you support on environment, tools, food etc?
Will it be on time to act when it becomes August?

Currently we already facing a recession. I believe an global outbreak on a more heavily level as it is now will empower the recession more. So more people will face uncertain conditions, more organizations will act on the next level of recession.

Are you already adapting your development processes based on this situation? Will their come a shift in processes?

Perhaps the world Matthew Heusser invented becomes true and needed: The Boutique Tester
Perhaps the isolated world I was thinking of becomes true: Testing in an Isolated World

Are you already start thinking about possible solutions? At least I'm already doing this, now wondering how we can take it to the next level. I hope my predictions don’t come out and we all work nice together.

Magazine: Quality Matters: Skill Based Testing

Today I'm proud to mention that my second article is published in free testing magazin Quality-Matters


This time a story about Skill Based Testing: The Life Cycle of a Tester.

I already wrote on this on this subject
December 8th 2008: Life Cycle of a Tester
July 1st 2009: Testing in an Isolated World

Testing in an Isolated World

As one of my favorite quotes comes from John Maynard Keynes (English economist, journalist, and financier, 1883-1946): "Ideas shape the course of history!"

Matthew Heusser has an idea. On his blog he posted an interesting concept The Boutique Tester: A tester as a craftsperson who adds value to the customer through evolution. He stated: "To compete as a craftsperson, the tester role will have to evolve. He'll have to be smarter, sharper, faster. In the boutique world, he will have to explain his services to people who are skeptical of such services and believe they can do it themselves."

Matt is "inventing" a future. He hopes there is place for the boutique tester and craftsmanship returns. He also thinks there is no room for it.

So I will try to help him. For this I am creating the idea because testers are needed due to an isolated world. This seems quite contradictive to globalizing of development and testing. We continue outsourcing, on- and off-shoring, etc. Mainly this is how organizations are processes; this results that also the activities are moved. Only this is not their main objective, it is a result.

The isolated world I'm talking about can be caused by global circumstances and the business needs to continue. For example: when the pandemia like the "swine-flu" is getting worse and people have to stay at home, is there any option for businesses just to stop producing? Of course there are other companies who are willing to take this task and claiming they are able to continue deliverance. Are they honest about this?

This week I read in the newspaper if the Dutch government didn't buy vaccines against the flu and people are not using the vaccines there is a probably chance of >30% of the people who will get the flu. I'm sure your boss cannot select people who might have the flu and who not because it will danger the productivity.

Perhaps in certain situations activities can be handed over easily to other people. I don't think it is completely possible in software testing. So there must be people who are skilled and created an environment where they are able to do the job and able to help customers although the "environment" is not available.

James Bach also replied on this idea on Have Internet, Will Test he gives a good example that is already possible. As he lives on an island, it would be cheaper if he could express his craftsmanship from there. I think he would be able to get the same results if the necessary tools are available.

Another idea I had for (this type of) tester(s) is to accept that not all knowledge and skills might be available in just one person. Defining the need for value is important, you might pick your tester based on the needed value, accept there is some kind of life cycle also for testers.
A posting I wrote on December 8th, 2008, Life Cycle of a Tester

Thinking about such a world is one thing, creating it on paper is another. Still the world is not yet there. Some people might turn to Second Life and use this world to communicate. Virtualization might be one option.
Here some points I could think about when creating the world; you have to:
1. Find a reason for existence
2. Find a way to communicate
3. Find/create the necessary tools.
4. Find people who are able and willing to live in this world
5. Find other people and make people find you
6. Create mutual trust
7. Proof added value
8. Keep communicating and preserve transparency
9. Find time
10. ... (10 is a nice number, as most of the lists ending with 10 items, I leave this open for other thoughts)

Most of the points are already there. You just have to prepare and exploit them.
Like 1: a pandemia is a valid reason, business must go on and people are not that yet able to communicate and work with innovative technologies
2. As we can communicate with phone, video, twitter, there seems also tools which enables you to share ideas, language, pictures desktops etc.
3. If you are missing tools, find them, the internet is full of it, ask colleagues, or ask the boutique developer.
4. Share the idea and promote you world
5. There are already people who are visible on the internet, are you also? When it happens, this might be a very important source of communication
6. You have to be able to show your integrity, it is not about the money, it is about the value.
7. Be able to tell what you have done for the customer, not which activities, what made the product better acceptable for the customer
8. Be able to express what you are doing, thinking and need. Don't confuse this with nice fancy words; in this world there is lesser time to convince people.
9. Spend the time the customer needs and you need.
10. ....

Think about how it would be if you are forced to live in an isolated world. Why not be able to add value from your type of world to others.

Mentor in software testing

Growing in testing
Over the years I read books, I attended conferences, read blogs, participate on forums and talk with colleagues about software testing. I also started my own blog just to write down ideas or information. All this time I had the idea that there must be more I can do. I already heard about Test Oracles, coaches and mentors and decided I to see how I could find a mentor. This made me post a question on Test Republic: How to find your mentor?
On that question I got great responses which made me write it down in this blog.
I believe the question and responses it selves is an good example about mentoring. Sharing information based on questions, starting discussions, asking for clarification to get the proper context and leaving space to make up your own mind. To all who participated in this question: Thanks!

Coaching vs Mentoring
Across the responses I noticed that some talk about coaching and some about mentoring. Is this the same? On the internet I found an interesting post from Keith Rosen posted about this Coach vs. Mentor: What's the Difference?

Coach:
An expert on people and personal development. …. A coach works on the whole person and is multidimensional, rather than focusing only on what the person is already doing. The coaching relationship is built on choice rather than necessity.

Mentor:
An expert in a field, industry, or at a company who typically acts as an internal advisor. …. Often, mentors are not trained, and their guidance is based more on their experience rather than the skills or proficiencies needed to mentor. Often, the mentoring relationship is need-driven rather than driven by choice.

To my opinion the similarity in this is:
- both guide you- both are helping you in an objective manner;
- both are letting you make your own decisions;

The difference could be:
- a mentor has experience and skills in the same field of business and even a bit beyond, the questions from a mentor can help you to make the step a bit faster;
- a coach can also be a person without any knowledge of your field of expertise

This still leaves the space that a few coaches together can be your mentor.

The test mentor
A test mentor is:
- Not a person who tells you what to do.
- Not written or single direction information sharing (conferences/courses/workshops/seminars)
- A person who provides you challenges and possible directions to define your own opinion or not. That is your own choice and the mentor’s choice.

Finding the mentor
Although some mentioned that mentors shouldn’t only be persons; it can also be books, articles, blogs, conferences etc. I still believe the power of mentoring is interaction. This leaves me to the original question to see if there is a fixed process for finding a mentor.

As Pradeep Soundararajan mentioned in this post “It ain't a process. Or maybe,”.
I agree with him, you cannot narrow it down in specific defined actions.

I came up with the following process steps:
1. Be ware of your own knowledge skills
2. Identify people who have something to say and might have more to show
3. Get known to those people
4. Trigger them to challenge you
5. Be open to their suggestions (questions)
6. Perhaps you will be challenged on regular basis or not

As you see, there is lot of space left for interpretation.

Shrini Kulkarni showed some similar steps:
1. Look inword - what your personal ideology, personal value system and philosophy
2. Look outwards and around to seek someone who aligns with (1) and who appears to be more knowledgeable or someone who can possibly know more about the subject than you
3. Establish the relationship with the selected mentor as in (2)

Establish the relationship
The contributors of this question showed me some ways how they found their mentor. As what I understand it can be hard: a lot of e-mails to be send; and also easy: You might be introduced.

It seems to be easy to find the names of mentors you think can help you. I have the impression that you are not alone who approach them. So you have to do more. You have to impress them and contact them on the right time. Their time is also valuable for them and of course they don’t want to spoil it either. Pradeep mentioned he got some challenges to beat. Matthew Heusser told about some heavy “black-belt” challenges. He uses this to make a shifting. And no, this is not an idea to introduce new certifications in mentoring like Six-Sigma. It is used as example within a certain context.

As far as I see it you have to be careful who you pick out as mentor. You also have to be ready. When taking mentoring too light you will waste either’s time.

When the time is right
This might differ for everyone. I noticed that the time might be right quite soon. I based this statement on:
- Leaving the methods I have been taught alone and use it as a reference to my experience;
- I started to make up my own ideas;
- I’m reading/listening between lines and words and try to figure out how it (can) helps me and also what feeling certain thought gives me;
- I’m willing to share my knowledge, ideas with others and willing to learn how it supports them.

The mentor and mentee
A relationship between mentor and mentee is not just one you have to take for granted. You must ask yourselves the question:”How far are you willing to go?” Pradeep Soundararajan pointed me at the following site: http://en.wikipedia.org/wiki/Ekalavya. This is a good example to keep in mind about the price are you willing to pay for mentorship.

Another aspect is the time which is spent. Not every one has the same amount of time or is available on the time you need.

What I see as a good example of mentoring is how Matt responded to my question. He picked some parts from the discussions and came up with an example how he would do it without saying this is how it must be done. Although he was quite clear, he leaves me space to continue thinking. I think this is one part of the relation. Ask question, respond with an example, accept this example and continue thinking about this trying to place it in other contexts, to come up with your own context.

Beside the obvious respect it is also important to be open for other contexts of thinking and try to learn how people are so you also understand their context. Sometimes words have other meanings then they were told.


What is mentoring after all?
Not knowing how to write it down in words
Mentoring is not just gaining information. It is sharing thoughts and creating thoughts within a broad spectrum.

Unattended mentoring can be done on several occasions like between colleagues while having a short break or at conferences were also a lot of people can be met. (I deliberately am not using words like: "formal" and "informal" mentoring as this gives the suggestion that there is a kind of contract.)

I would divide Attended mentoring into "Indirect" and "Direct" mentoring
Indirect mentoring: being available when the occasion is there
Direct Mentoring: be on scheduled terms or when needed

Though it is not a process with defined activities. It can be captured in few process steps were the outcome is not defined and should not be defined.

Driving a car successfully or driving a successfully car?

You might yourselves the question what you really are doing and what your objectives are.
What do you want: "Driving a car successfully or driving a successfully car?"
Or perhaps successfully driving a (successful) car?

Although this question has nothing to do with testing. It has also everything to do with testing. And no, this is not the metaphor about explicitly mentioning colors, requirements and what car you want. It should make you think how you approach your testing.

Introduction
I'm am aware that there are laws and rules on certain places which states you have to be above a certain age, you need a driving license, you need a car, preferably your own car. I also know that there are different perceptions about what successfully is and what cars are. And yes, they might have some influence on the outcome. And no, I won't consider them in the lines which will follow. Sometimes it is better to leave certain values and restrictions to keep a better picture and stick to a main concept.

Purpose
I believe it is in human nature when they have to do something they start doing their best. They want to be successfully. I think it is also in human nature to give own interpretations to questions, assignments, stories etc. These interpretations are mainly created based on information which is not verbally given. When someone hires you for testing, you assume you should test like you always did. You start using your best practices without checking if that is wanted.

The same can be done with driving a car. A purpose of driving a car which makes sense is to go from point A to point B.

I think here the mistake starts. Based on best practices you can assume that from point A to point B should be done in limited time. Did you check this? Or should it be done in an optimal way. What is an optimal way? The shortest route? The less gasoline consuming route?

Perhaps you even don't look for information and you sit in your car and based on your experience you go beyond the speed limits. Is this asked? Is this allowed?
Are these the right questions?

Driving a car successfully
Imagine you have your car and start driving. Without instructions and the assignment you are willing to go to point B. As you are an experience driver you hit the road. During the trip you rely on your best practices. You were taught to watch the road signs, monitor the gasoline meter, look further then just the car before you, use a kind of mapping device, or map, (or perhaps you know were point B is and you don't use them at all.), etc. You make assumptions based on your interpretation and you take action according your best practices. Your reach point B.

You might think I drove my car successfully as I made it to point B without accidents, within time using the allowed gasoline.

Are you really successful? What about:
- the location of point A?
- is getting from A to B the main purpose? Or can it also be taking some type of luggage in the direction of point B?
- is point B a coordinate on a map or just a reference to a location?
- are you sure that the care must be unharmed?
- was reaching point B the objective? Perhaps it was the task to do everything but avoid point B.

How did your best practices influence your decisions and would it help if you first started with investigating the main purpose and conditions? Don't interpreted this that all requirements should be clear and SMART. Only those who matters.

Do you also start approaching a project as written in TMap or ISTQB? Because your references and best practices are based on these ideas? Or do you investigate and then decide of a method like this can be of some use?

Driving a successfully car
A Porsche is a great car. So are Lamborghini, Ferrari, VW, Skoda and so on. This is what we should believe if we read all the adds, articles, forums, researches. And all can be used for driving. If you believe unconditionally all those stories, adds etc, you are driving a successfully car. At least you believe this. So will everyone have his/her successfully car. Are you willing to drive the car of your boss although it is not your favorite? Be careful how to respond as it is the boss's successfully car. Or is it? Perhaps your boss was also was forced to drive it.

To decide which car is a successful care for you, you interpreted all kinds of information and you based your point of view on it. It helped to built up your experience about driving this car. Somehow it is also in human nature to defend your believe about a successful car, of course it is yours otherwise you wouldn't driving it.

There might be some kind of process behind:
You first bought a car for a purpose; you start some investigation which car suites the best.

You are aware that there are better cars, only you cannot afford them or simply not getting them as they are rare in your region. After a while you start believing that your car is also successful based on your experience. Your car gives you joy and a good feeling. Finally you think you know that your car is the only successful car as you make the assumption that there is a relation between you being successful and your car. You are no longer open for discussion on terms which matters and using the cruise control to drive.

In the world of software testing this is also happening. How much adds, articles etc are not yet written that course like ISTQB, TMap, TMMi, Requirements engineering etc are "promising" that you must have those certifications and work according the written content? Don't you think that those organizations are trying convincing that they are the only successful tool to keep you driving? Do you ask yourselves every time if the conditions which can lead to success is also available and necessary to do your driving job?

Successfully driving a (successful) car
Driving your car under the given conditions reaching the objectives you were given can lead to success. To drive in a car which you prefer and gives you joy might lead to a successful car to support your trip. This seems to be a win-win situation. In this my advice will be also watching the environment you are driving in.

Sometimes you see opportunities which make it worth to stop and shoot some pictures. Sometimes you are challenged to speed up and breaking some rules against speed limit. Although it is not allowed, it can bring you on other thoughts; it helps you to see things in other perspectives. Also slowing down might support your trip and vision. On your trip there are always some kind of unpredicted situations triggered by the "unknown-unknowns" it helps if you change behavior. Perhaps you might not reached your goal within the agreed terms. It can help you to add value to your trip.

In testing it can be the same. If you decided to go on the road with given means and methods take time to judge those and check if they are still given the value needed. Perhaps other value can also be added at that moment. Dare to question your approach and don't rely on the cruise control of your best practices with test methods.

Or just driving?
Another option can be just starting driving. Just see were it brings you. This can be useful and fun. Do something you normally don't do. Don't make it as the daily trip to your job. When opening your mind for circumstances you might find opportunities to help using the means you have.

This is something different then approaching a project with your best practices and methods because there is not yet some direction defined. If you introduce those you will directly have impact on the route the project will go. In this situation it not your skill which will add value to the project; is the method which might define unnecessary conditions towards the project and make it less successful. Sometimes it is better to walk around and see how you can use your skills. Using your skills is not forcing your skills to be used.

Conclusion
Success is not based on methods and best practices, they might help. Written words can be misleading. There is always a context behind a question/assignment. Don't just do what you have been taught to do. Be aware that changing behavior can help to set everything it the proper perspective.

Weather forecast and testing

Testing is like predicting the weather.
Some of the following similarities can be found:
- When it is raining, bugs are hiding, so before start testing first makes the rain stop;
- When it starts raining, look carefully were the bugs are running to. To catch them, tools might be needed to avoid you to destruct their hiding place;
- Based on current values we compare them with historical data and try to predict the next behavior of the next day, days, weeks;
- In both worlds fancy charts are created and only the experienced ones can understand them;
- The next day might differ the prediction, you can tell the weather will be nice, when the moment comes we monitor if the prediction comes out;
- If bad weather is expected we take measurements;
- Even if bad weather is expected, we have to go outside;
- The environmental circumstances can have huge influence on the weather;
- If predictions become valid, enjoy that moment and continue;
- Use the proper tools for the right purpose. A wind meter will not measure the rainfall;
- Gaining weather information can be done manually and also automatically;
- Sometimes it is cheaper and reliable to stick your head outside the window to see what the weather is doing;
- The audience differs, some people are waiting for rain, some for just dry weather and some are interested in the clouds;
- Weather is a concept with different meanings, so is testing.

I'm certain that there are more similarities between weather forecasting and testing. I would say: take nothing for granted and ask yourselves what you would do in certain situations after translating it to weather conditions.

Users as testers

Not every one who reads books wants to write books. Not every one who writes books read books from others. Not every writer is able to write in another genre and specialism. Not every one who wants to write books are good at it.

If this is true, why do we often expect users to start testing applications and trust on their verdict? Why do we rely on that outcome if we know we never told them what testing is and how to test. What we expect them to do?

I think it is because we trust them based on what they have shown. I also think a misperception is made based on wrong behavior. You can say that someone who performs one test is a tester with added value, therefore you can’t say: some user who performs a test is a tester with added value.
Sure, the users know how the system works and are able to tell what they do and how they have done it. Is this enough?

You have readers who are able to re-tell what they have read. There are also readers who are able to write a small review on that book. Still they are not able to write a similar book which attracts the same audience.

There are also writers who are able to write books for their audience. If they are asked to write for a newer audience they have to start learning again.

Can users be good testers? I think they can, only you have to be aware which value you are requesting from them. Don't expect a reader to become a writer in one day. Don't expect a user to be a good tester from the beginning. In both situations you have to guide them.

Also don't expect a tester to be a good user.

There is a Software Testing Manifesto

I was a bit surprised when I noticed there is a site which claims to be a manifesto from software testing: Software Testing Manifesto. On this site they present the outcome of several workshops during EuroStar 2008 related to this topic. I think it is a good idea to share information. Others might learn from it, in a positive of negative way.

The bad thing is how that information is presented. When calling it a manifesto you assume this is the truth and we all could live by those principles. Looking at the URL "http://www.softwaretestingmanifesto.org/" to me it implies it is an independent statement, although there is a large advertisement of EuroStar on it. Second: it assumes that all testers comply with it.

Also the owner of this site is related to another software testing related company, I wonder if the site is moderated with objectivity.
Although the whole internet community can vote for their favorite statements and there is an option to pose new statements, it is unclear to me how statements are moderated.

From WikiPedia the definition of Manifesto is: "A manifesto is a public declaration of principles and intentions, often political in nature, but may also be life stance related. However, manifestos relating to religious belief are rather referred to as credo."

The name of the site and the name of the manifesto suggest that all software testers should comply to this manifesto. I think that is quite dangerous. As I already said I appreciate information sharing only the context should also be made clearer. It would suite them better that this site was a manifesto with an eye wink and explicitly mentioned the purpose of this site. A goal for me would be more: "do we need a general manifesto for software testing and start the discussion, then confronting the whole population with software testers that they have to comply with these statements.

Perhaps I'm wrong about this initiative. Currently I see this initiative as a dangerous one.

In the twilight zone of testing

Sometimes you have that feeling that the time is there to make a step forward. You have the feeling there must be more. Currently I'm also at this moment. I call it my own twilight zone of testing. James Bach triggered me to write this posting due to his post: Reclaim Your Personal Method.

Over the years I got familiar with test methods like TMap and ISTQB. In the last years I started to read more on the internet what other testers can teach me. Actually I was more looking for guidelines and guides who might help me to lead me on the path I like to wander.

When looking back I think you can identify the following phases:
- Knowledge phase: a period were you gain you basic and initial knowledge
- Awareness phase: a period were you become aware you have some knowledge
- Denial phase: a period you claim your knowledge is sufficient
- Consolidation phase: a period were you keep using your gained knowledge and start playing with other ideas, methods etc in your approach
- Innovation phase: A period were you are familiar with adapting other ideas, approaches, methods and make your own method.
- Destruction phase: This might be a period which should be avoided. A period were you stop learning and adapting

In my opinion these phases don't have solid boundaries. You will move sometimes unattended in the next phase. Sometimes you might have the feeling you know you are ready to make the next step only the guiding path is missing or just that extra push. It seems to be easy to just hop over to the next phase. Only you need more: you need confidence and trust in your capabilities with the danger not to exaggerate.

Currently I feel I'm in that twilight zone, ready to make the next jump, waiting for the chance to make the jump and working on opportunities which make me feel suitable to be able to succeed in the next phase.

For that time being, I'm keep on writing on this blog and reading others work and discussing with people and when needed ask completely strangers to provide me some help.

Fun: Black Box Test Machines

Recently I found this site were some example programs are made available to
Black Box Test Machines by Workroom productions from James Lyndsay.

Quoted from his site: "The following freeware machines are testing puzzles - crosswords for testers. Open one up and play with it. Try to answer these three questions: What's it doing? What's it doing wrong? How can I be so sure?"

When you have some spare time you might take a look at those applications and try to answer the questions. It is also worthwhile to read his award winning paper: Adventures in Session-based Testing"

Model Based Testing was this new?

I never would claim I have the knowledge. I have some knowledge and are always eager to learn. Last year I attended some conferences and one hot topic was Model Based Testing. It promised to be new and useful.

Searching for some more information about this topic I ran into this article: Model-Based Testing in Practice by S.R. Dalal, A. Jain, N. Karunanithi, J.M. Leaton, C.M. Lott, G.C.Patton, B.M.Horowitz.

Looking at the date of this article it is already from 1999. The strong part of this article I think is the focus on the method it selves instead of using tools.

When looking to recently published information the focus lies more on usage of tools. Some examples:
At EuroStar 2008 it was also a topic.

Experiences from working with Model Based Testing (MBT) and Qtronic) by Hakan Fredriksson. Remarkable is in this presentation the notation towards Rogers Adoption /Innovation curve. He shows that organizations who using MBT are early adopters. (Although it is already old for a decade)

Testing Experience: second issue 2009: "MBT as the next step in testing" by Elise Greveraars. Also her presentation on EuroStar 2008 went about how to use tools in model based testing: "Tester Needed? No Thanks we use MBT"

An interesting video by Mark Utting (August 2007) shows also an approach for MBT using tools: http://video.google.com/videoplay?docid=5521890509476590796

Quotes to think about

Recently, driving in my car to my assignement I heard a commercial on the radio where they mentioned a quote from John Maynard Keynes (English economist, journalist, and financier, 1883-1946): "Ideas shape the course of history!"

I loved that quote as I'm full of ideas. Perhaps I can shape the course some day :)

This triggered me to search for a few other quotes:

G. Moore: "Everybody sets out to do something, and everybody does something, but no one does what he sets out to do."

B. Shaw (Caesar & Cleopatra (1898): "When a stupid man is doing something he is ashamed of, he always declares that it is his duty."

J. Dryden (All for love (1976): "Errors, like straws, upon the surface flow; he would search for pearls must dive below."

A. Tennyson: "Sweet is it have done the thing one ought."

John Updike (The Carp (1978): "An expert takes nothing personally. Nothing is even precisely his fault. If a bridge collapses or a war miscarries, he has already walked away. He still has his expertise."

Blocking your nurture

What do you do when a person ask you a question? Do you try to answer it immediately or ask an own question to get more information about the purpose?
What do you do when a person explains a problem? Do you try to solve it or ask form more information?

It is in my nature to help people. If a person asks me the time I look at my watch and tells him what is on my watch. If I forgot my watch I look around to check if I see a clock. If no means are available to tell the exact time I try to estimate the time. It is so simple and obvious to respond automatically to triggers we have done before using experience and knowledge we gained before.

The question is: did this help the person? When someone asks for the time and you give the time it seems that the question is answered. Still there might be more behind that question. Perhaps that person used that question to make contact. If you give the time, the chance for further contact is minimized. If that person needed the time to check whether he is late only he is not able to calculate then the question is obviously answered only the person cannot do anything with it. Perhaps he actually wanted to ask if he is late for a certain appointment related to the current time.

The same might happen in testing, when a person tells you that his products is lacking quality you can tell him to test better. Nowadays it normal that in the same sentence we tell them we are able to do that part of testing and inform him that we are able to use methods for this like TMap, ISEB/ISTQB, TestGoal, CDT, SmarTest or whatever. Do to our experience it became our nurture to tell about our skills and experience.

Telling about skills and experience won't help a person. Using them might help. Only it can only help if you know it will fit the problem/ question. Before you start using your skills/experience about methods you first need to check if it is sufficient. It is important to start gaining information first. Then you can decide which solution/method is the best for helping the person.

I think here the tricky part comes: It is in our nurture to start asking questions based on our knowledge and experience. If that knowledge is restricted then the gained information is also restricted. Perhaps it is useful for the method you like to use; the pitfall here is that you might be forced to change customer’s processes and make them accept the disadvantages to make your method work.

I don't know exactly how to resolve this problem. At least I believe sticking to the knowledge/rules of one method is insufficient. Also the knowledge of 2 similar methods like ISTQB and TMap won't work. You will be forced thinking in a certain pattern although this pattern cannot be the answer. It might help you to lead to a solution. Currently I'm reading more beyond these approaches, listening to ideas of others and trying to make up my own ideas. I'm still aware I'm at the beginning of the journey where it helps to start with the statement: Currently no method is useful, let us first see what want to do and what we are currently doing.