Mentor in software testing

Growing in testing
Over the years I read books, I attended conferences, read blogs, participate on forums and talk with colleagues about software testing. I also started my own blog just to write down ideas or information. All this time I had the idea that there must be more I can do. I already heard about Test Oracles, coaches and mentors and decided I to see how I could find a mentor. This made me post a question on Test Republic: How to find your mentor?
On that question I got great responses which made me write it down in this blog.
I believe the question and responses it selves is an good example about mentoring. Sharing information based on questions, starting discussions, asking for clarification to get the proper context and leaving space to make up your own mind. To all who participated in this question: Thanks!

Coaching vs Mentoring
Across the responses I noticed that some talk about coaching and some about mentoring. Is this the same? On the internet I found an interesting post from Keith Rosen posted about this Coach vs. Mentor: What's the Difference?

Coach:
An expert on people and personal development. …. A coach works on the whole person and is multidimensional, rather than focusing only on what the person is already doing. The coaching relationship is built on choice rather than necessity.

Mentor:
An expert in a field, industry, or at a company who typically acts as an internal advisor. …. Often, mentors are not trained, and their guidance is based more on their experience rather than the skills or proficiencies needed to mentor. Often, the mentoring relationship is need-driven rather than driven by choice.

To my opinion the similarity in this is:
- both guide you- both are helping you in an objective manner;
- both are letting you make your own decisions;

The difference could be:
- a mentor has experience and skills in the same field of business and even a bit beyond, the questions from a mentor can help you to make the step a bit faster;
- a coach can also be a person without any knowledge of your field of expertise

This still leaves the space that a few coaches together can be your mentor.

The test mentor
A test mentor is:
- Not a person who tells you what to do.
- Not written or single direction information sharing (conferences/courses/workshops/seminars)
- A person who provides you challenges and possible directions to define your own opinion or not. That is your own choice and the mentor’s choice.

Finding the mentor
Although some mentioned that mentors shouldn’t only be persons; it can also be books, articles, blogs, conferences etc. I still believe the power of mentoring is interaction. This leaves me to the original question to see if there is a fixed process for finding a mentor.

As Pradeep Soundararajan mentioned in this post “It ain't a process. Or maybe,”.
I agree with him, you cannot narrow it down in specific defined actions.

I came up with the following process steps:
1. Be ware of your own knowledge skills
2. Identify people who have something to say and might have more to show
3. Get known to those people
4. Trigger them to challenge you
5. Be open to their suggestions (questions)
6. Perhaps you will be challenged on regular basis or not

As you see, there is lot of space left for interpretation.

Shrini Kulkarni showed some similar steps:
1. Look inword - what your personal ideology, personal value system and philosophy
2. Look outwards and around to seek someone who aligns with (1) and who appears to be more knowledgeable or someone who can possibly know more about the subject than you
3. Establish the relationship with the selected mentor as in (2)

Establish the relationship
The contributors of this question showed me some ways how they found their mentor. As what I understand it can be hard: a lot of e-mails to be send; and also easy: You might be introduced.

It seems to be easy to find the names of mentors you think can help you. I have the impression that you are not alone who approach them. So you have to do more. You have to impress them and contact them on the right time. Their time is also valuable for them and of course they don’t want to spoil it either. Pradeep mentioned he got some challenges to beat. Matthew Heusser told about some heavy “black-belt” challenges. He uses this to make a shifting. And no, this is not an idea to introduce new certifications in mentoring like Six-Sigma. It is used as example within a certain context.

As far as I see it you have to be careful who you pick out as mentor. You also have to be ready. When taking mentoring too light you will waste either’s time.

When the time is right
This might differ for everyone. I noticed that the time might be right quite soon. I based this statement on:
- Leaving the methods I have been taught alone and use it as a reference to my experience;
- I started to make up my own ideas;
- I’m reading/listening between lines and words and try to figure out how it (can) helps me and also what feeling certain thought gives me;
- I’m willing to share my knowledge, ideas with others and willing to learn how it supports them.

The mentor and mentee
A relationship between mentor and mentee is not just one you have to take for granted. You must ask yourselves the question:”How far are you willing to go?” Pradeep Soundararajan pointed me at the following site: http://en.wikipedia.org/wiki/Ekalavya. This is a good example to keep in mind about the price are you willing to pay for mentorship.

Another aspect is the time which is spent. Not every one has the same amount of time or is available on the time you need.

What I see as a good example of mentoring is how Matt responded to my question. He picked some parts from the discussions and came up with an example how he would do it without saying this is how it must be done. Although he was quite clear, he leaves me space to continue thinking. I think this is one part of the relation. Ask question, respond with an example, accept this example and continue thinking about this trying to place it in other contexts, to come up with your own context.

Beside the obvious respect it is also important to be open for other contexts of thinking and try to learn how people are so you also understand their context. Sometimes words have other meanings then they were told.


What is mentoring after all?
Not knowing how to write it down in words
Mentoring is not just gaining information. It is sharing thoughts and creating thoughts within a broad spectrum.

Unattended mentoring can be done on several occasions like between colleagues while having a short break or at conferences were also a lot of people can be met. (I deliberately am not using words like: "formal" and "informal" mentoring as this gives the suggestion that there is a kind of contract.)

I would divide Attended mentoring into "Indirect" and "Direct" mentoring
Indirect mentoring: being available when the occasion is there
Direct Mentoring: be on scheduled terms or when needed

Though it is not a process with defined activities. It can be captured in few process steps were the outcome is not defined and should not be defined.

Driving a car successfully or driving a successfully car?

You might yourselves the question what you really are doing and what your objectives are.
What do you want: "Driving a car successfully or driving a successfully car?"
Or perhaps successfully driving a (successful) car?

Although this question has nothing to do with testing. It has also everything to do with testing. And no, this is not the metaphor about explicitly mentioning colors, requirements and what car you want. It should make you think how you approach your testing.

Introduction
I'm am aware that there are laws and rules on certain places which states you have to be above a certain age, you need a driving license, you need a car, preferably your own car. I also know that there are different perceptions about what successfully is and what cars are. And yes, they might have some influence on the outcome. And no, I won't consider them in the lines which will follow. Sometimes it is better to leave certain values and restrictions to keep a better picture and stick to a main concept.

Purpose
I believe it is in human nature when they have to do something they start doing their best. They want to be successfully. I think it is also in human nature to give own interpretations to questions, assignments, stories etc. These interpretations are mainly created based on information which is not verbally given. When someone hires you for testing, you assume you should test like you always did. You start using your best practices without checking if that is wanted.

The same can be done with driving a car. A purpose of driving a car which makes sense is to go from point A to point B.

I think here the mistake starts. Based on best practices you can assume that from point A to point B should be done in limited time. Did you check this? Or should it be done in an optimal way. What is an optimal way? The shortest route? The less gasoline consuming route?

Perhaps you even don't look for information and you sit in your car and based on your experience you go beyond the speed limits. Is this asked? Is this allowed?
Are these the right questions?

Driving a car successfully
Imagine you have your car and start driving. Without instructions and the assignment you are willing to go to point B. As you are an experience driver you hit the road. During the trip you rely on your best practices. You were taught to watch the road signs, monitor the gasoline meter, look further then just the car before you, use a kind of mapping device, or map, (or perhaps you know were point B is and you don't use them at all.), etc. You make assumptions based on your interpretation and you take action according your best practices. Your reach point B.

You might think I drove my car successfully as I made it to point B without accidents, within time using the allowed gasoline.

Are you really successful? What about:
- the location of point A?
- is getting from A to B the main purpose? Or can it also be taking some type of luggage in the direction of point B?
- is point B a coordinate on a map or just a reference to a location?
- are you sure that the care must be unharmed?
- was reaching point B the objective? Perhaps it was the task to do everything but avoid point B.

How did your best practices influence your decisions and would it help if you first started with investigating the main purpose and conditions? Don't interpreted this that all requirements should be clear and SMART. Only those who matters.

Do you also start approaching a project as written in TMap or ISTQB? Because your references and best practices are based on these ideas? Or do you investigate and then decide of a method like this can be of some use?

Driving a successfully car
A Porsche is a great car. So are Lamborghini, Ferrari, VW, Skoda and so on. This is what we should believe if we read all the adds, articles, forums, researches. And all can be used for driving. If you believe unconditionally all those stories, adds etc, you are driving a successfully car. At least you believe this. So will everyone have his/her successfully car. Are you willing to drive the car of your boss although it is not your favorite? Be careful how to respond as it is the boss's successfully car. Or is it? Perhaps your boss was also was forced to drive it.

To decide which car is a successful care for you, you interpreted all kinds of information and you based your point of view on it. It helped to built up your experience about driving this car. Somehow it is also in human nature to defend your believe about a successful car, of course it is yours otherwise you wouldn't driving it.

There might be some kind of process behind:
You first bought a car for a purpose; you start some investigation which car suites the best.

You are aware that there are better cars, only you cannot afford them or simply not getting them as they are rare in your region. After a while you start believing that your car is also successful based on your experience. Your car gives you joy and a good feeling. Finally you think you know that your car is the only successful car as you make the assumption that there is a relation between you being successful and your car. You are no longer open for discussion on terms which matters and using the cruise control to drive.

In the world of software testing this is also happening. How much adds, articles etc are not yet written that course like ISTQB, TMap, TMMi, Requirements engineering etc are "promising" that you must have those certifications and work according the written content? Don't you think that those organizations are trying convincing that they are the only successful tool to keep you driving? Do you ask yourselves every time if the conditions which can lead to success is also available and necessary to do your driving job?

Successfully driving a (successful) car
Driving your car under the given conditions reaching the objectives you were given can lead to success. To drive in a car which you prefer and gives you joy might lead to a successful car to support your trip. This seems to be a win-win situation. In this my advice will be also watching the environment you are driving in.

Sometimes you see opportunities which make it worth to stop and shoot some pictures. Sometimes you are challenged to speed up and breaking some rules against speed limit. Although it is not allowed, it can bring you on other thoughts; it helps you to see things in other perspectives. Also slowing down might support your trip and vision. On your trip there are always some kind of unpredicted situations triggered by the "unknown-unknowns" it helps if you change behavior. Perhaps you might not reached your goal within the agreed terms. It can help you to add value to your trip.

In testing it can be the same. If you decided to go on the road with given means and methods take time to judge those and check if they are still given the value needed. Perhaps other value can also be added at that moment. Dare to question your approach and don't rely on the cruise control of your best practices with test methods.

Or just driving?
Another option can be just starting driving. Just see were it brings you. This can be useful and fun. Do something you normally don't do. Don't make it as the daily trip to your job. When opening your mind for circumstances you might find opportunities to help using the means you have.

This is something different then approaching a project with your best practices and methods because there is not yet some direction defined. If you introduce those you will directly have impact on the route the project will go. In this situation it not your skill which will add value to the project; is the method which might define unnecessary conditions towards the project and make it less successful. Sometimes it is better to walk around and see how you can use your skills. Using your skills is not forcing your skills to be used.

Conclusion
Success is not based on methods and best practices, they might help. Written words can be misleading. There is always a context behind a question/assignment. Don't just do what you have been taught to do. Be aware that changing behavior can help to set everything it the proper perspective.

Weather forecast and testing

Testing is like predicting the weather.
Some of the following similarities can be found:
- When it is raining, bugs are hiding, so before start testing first makes the rain stop;
- When it starts raining, look carefully were the bugs are running to. To catch them, tools might be needed to avoid you to destruct their hiding place;
- Based on current values we compare them with historical data and try to predict the next behavior of the next day, days, weeks;
- In both worlds fancy charts are created and only the experienced ones can understand them;
- The next day might differ the prediction, you can tell the weather will be nice, when the moment comes we monitor if the prediction comes out;
- If bad weather is expected we take measurements;
- Even if bad weather is expected, we have to go outside;
- The environmental circumstances can have huge influence on the weather;
- If predictions become valid, enjoy that moment and continue;
- Use the proper tools for the right purpose. A wind meter will not measure the rainfall;
- Gaining weather information can be done manually and also automatically;
- Sometimes it is cheaper and reliable to stick your head outside the window to see what the weather is doing;
- The audience differs, some people are waiting for rain, some for just dry weather and some are interested in the clouds;
- Weather is a concept with different meanings, so is testing.

I'm certain that there are more similarities between weather forecasting and testing. I would say: take nothing for granted and ask yourselves what you would do in certain situations after translating it to weather conditions.

Users as testers

Not every one who reads books wants to write books. Not every one who writes books read books from others. Not every writer is able to write in another genre and specialism. Not every one who wants to write books are good at it.

If this is true, why do we often expect users to start testing applications and trust on their verdict? Why do we rely on that outcome if we know we never told them what testing is and how to test. What we expect them to do?

I think it is because we trust them based on what they have shown. I also think a misperception is made based on wrong behavior. You can say that someone who performs one test is a tester with added value, therefore you can’t say: some user who performs a test is a tester with added value.
Sure, the users know how the system works and are able to tell what they do and how they have done it. Is this enough?

You have readers who are able to re-tell what they have read. There are also readers who are able to write a small review on that book. Still they are not able to write a similar book which attracts the same audience.

There are also writers who are able to write books for their audience. If they are asked to write for a newer audience they have to start learning again.

Can users be good testers? I think they can, only you have to be aware which value you are requesting from them. Don't expect a reader to become a writer in one day. Don't expect a user to be a good tester from the beginning. In both situations you have to guide them.

Also don't expect a tester to be a good user.

There is a Software Testing Manifesto

I was a bit surprised when I noticed there is a site which claims to be a manifesto from software testing: Software Testing Manifesto. On this site they present the outcome of several workshops during EuroStar 2008 related to this topic. I think it is a good idea to share information. Others might learn from it, in a positive of negative way.

The bad thing is how that information is presented. When calling it a manifesto you assume this is the truth and we all could live by those principles. Looking at the URL "http://www.softwaretestingmanifesto.org/" to me it implies it is an independent statement, although there is a large advertisement of EuroStar on it. Second: it assumes that all testers comply with it.

Also the owner of this site is related to another software testing related company, I wonder if the site is moderated with objectivity.
Although the whole internet community can vote for their favorite statements and there is an option to pose new statements, it is unclear to me how statements are moderated.

From WikiPedia the definition of Manifesto is: "A manifesto is a public declaration of principles and intentions, often political in nature, but may also be life stance related. However, manifestos relating to religious belief are rather referred to as credo."

The name of the site and the name of the manifesto suggest that all software testers should comply to this manifesto. I think that is quite dangerous. As I already said I appreciate information sharing only the context should also be made clearer. It would suite them better that this site was a manifesto with an eye wink and explicitly mentioned the purpose of this site. A goal for me would be more: "do we need a general manifesto for software testing and start the discussion, then confronting the whole population with software testers that they have to comply with these statements.

Perhaps I'm wrong about this initiative. Currently I see this initiative as a dangerous one.

In the twilight zone of testing

Sometimes you have that feeling that the time is there to make a step forward. You have the feeling there must be more. Currently I'm also at this moment. I call it my own twilight zone of testing. James Bach triggered me to write this posting due to his post: Reclaim Your Personal Method.

Over the years I got familiar with test methods like TMap and ISTQB. In the last years I started to read more on the internet what other testers can teach me. Actually I was more looking for guidelines and guides who might help me to lead me on the path I like to wander.

When looking back I think you can identify the following phases:
- Knowledge phase: a period were you gain you basic and initial knowledge
- Awareness phase: a period were you become aware you have some knowledge
- Denial phase: a period you claim your knowledge is sufficient
- Consolidation phase: a period were you keep using your gained knowledge and start playing with other ideas, methods etc in your approach
- Innovation phase: A period were you are familiar with adapting other ideas, approaches, methods and make your own method.
- Destruction phase: This might be a period which should be avoided. A period were you stop learning and adapting

In my opinion these phases don't have solid boundaries. You will move sometimes unattended in the next phase. Sometimes you might have the feeling you know you are ready to make the next step only the guiding path is missing or just that extra push. It seems to be easy to just hop over to the next phase. Only you need more: you need confidence and trust in your capabilities with the danger not to exaggerate.

Currently I feel I'm in that twilight zone, ready to make the next jump, waiting for the chance to make the jump and working on opportunities which make me feel suitable to be able to succeed in the next phase.

For that time being, I'm keep on writing on this blog and reading others work and discussing with people and when needed ask completely strangers to provide me some help.

Fun: Black Box Test Machines

Recently I found this site were some example programs are made available to
Black Box Test Machines by Workroom productions from James Lyndsay.

Quoted from his site: "The following freeware machines are testing puzzles - crosswords for testers. Open one up and play with it. Try to answer these three questions: What's it doing? What's it doing wrong? How can I be so sure?"

When you have some spare time you might take a look at those applications and try to answer the questions. It is also worthwhile to read his award winning paper: Adventures in Session-based Testing"

Model Based Testing was this new?

I never would claim I have the knowledge. I have some knowledge and are always eager to learn. Last year I attended some conferences and one hot topic was Model Based Testing. It promised to be new and useful.

Searching for some more information about this topic I ran into this article: Model-Based Testing in Practice by S.R. Dalal, A. Jain, N. Karunanithi, J.M. Leaton, C.M. Lott, G.C.Patton, B.M.Horowitz.

Looking at the date of this article it is already from 1999. The strong part of this article I think is the focus on the method it selves instead of using tools.

When looking to recently published information the focus lies more on usage of tools. Some examples:
At EuroStar 2008 it was also a topic.

Experiences from working with Model Based Testing (MBT) and Qtronic) by Hakan Fredriksson. Remarkable is in this presentation the notation towards Rogers Adoption /Innovation curve. He shows that organizations who using MBT are early adopters. (Although it is already old for a decade)

Testing Experience: second issue 2009: "MBT as the next step in testing" by Elise Greveraars. Also her presentation on EuroStar 2008 went about how to use tools in model based testing: "Tester Needed? No Thanks we use MBT"

An interesting video by Mark Utting (August 2007) shows also an approach for MBT using tools: http://video.google.com/videoplay?docid=5521890509476590796

Quotes to think about

Recently, driving in my car to my assignement I heard a commercial on the radio where they mentioned a quote from John Maynard Keynes (English economist, journalist, and financier, 1883-1946): "Ideas shape the course of history!"

I loved that quote as I'm full of ideas. Perhaps I can shape the course some day :)

This triggered me to search for a few other quotes:

G. Moore: "Everybody sets out to do something, and everybody does something, but no one does what he sets out to do."

B. Shaw (Caesar & Cleopatra (1898): "When a stupid man is doing something he is ashamed of, he always declares that it is his duty."

J. Dryden (All for love (1976): "Errors, like straws, upon the surface flow; he would search for pearls must dive below."

A. Tennyson: "Sweet is it have done the thing one ought."

John Updike (The Carp (1978): "An expert takes nothing personally. Nothing is even precisely his fault. If a bridge collapses or a war miscarries, he has already walked away. He still has his expertise."

Blocking your nurture

What do you do when a person ask you a question? Do you try to answer it immediately or ask an own question to get more information about the purpose?
What do you do when a person explains a problem? Do you try to solve it or ask form more information?

It is in my nature to help people. If a person asks me the time I look at my watch and tells him what is on my watch. If I forgot my watch I look around to check if I see a clock. If no means are available to tell the exact time I try to estimate the time. It is so simple and obvious to respond automatically to triggers we have done before using experience and knowledge we gained before.

The question is: did this help the person? When someone asks for the time and you give the time it seems that the question is answered. Still there might be more behind that question. Perhaps that person used that question to make contact. If you give the time, the chance for further contact is minimized. If that person needed the time to check whether he is late only he is not able to calculate then the question is obviously answered only the person cannot do anything with it. Perhaps he actually wanted to ask if he is late for a certain appointment related to the current time.

The same might happen in testing, when a person tells you that his products is lacking quality you can tell him to test better. Nowadays it normal that in the same sentence we tell them we are able to do that part of testing and inform him that we are able to use methods for this like TMap, ISEB/ISTQB, TestGoal, CDT, SmarTest or whatever. Do to our experience it became our nurture to tell about our skills and experience.

Telling about skills and experience won't help a person. Using them might help. Only it can only help if you know it will fit the problem/ question. Before you start using your skills/experience about methods you first need to check if it is sufficient. It is important to start gaining information first. Then you can decide which solution/method is the best for helping the person.

I think here the tricky part comes: It is in our nurture to start asking questions based on our knowledge and experience. If that knowledge is restricted then the gained information is also restricted. Perhaps it is useful for the method you like to use; the pitfall here is that you might be forced to change customer’s processes and make them accept the disadvantages to make your method work.

I don't know exactly how to resolve this problem. At least I believe sticking to the knowledge/rules of one method is insufficient. Also the knowledge of 2 similar methods like ISTQB and TMap won't work. You will be forced thinking in a certain pattern although this pattern cannot be the answer. It might help you to lead to a solution. Currently I'm reading more beyond these approaches, listening to ideas of others and trying to make up my own ideas. I'm still aware I'm at the beginning of the journey where it helps to start with the statement: Currently no method is useful, let us first see what want to do and what we are currently doing.

Do you trust the Work-Around

In the past I already made a post related to workarounds March 23rd, 2008: Start managing your Workaround's!

In that posting I advice to start with documenting workarounds and managing them based on their lifecycle. We should avoid that workarounds get normal desired functionality by just using it; the next step would be that that usage become normal expected activities.

An article I read a while ago is related to functionality which is default in MS Outlook, auto complete e-mail addresses, only that functionality is often not embedded in processes. It is there, only organizations often don't tell you to use it or not.
CNet new: February 5th, 2008: Lilly's $1 Billion E-Mailstrom;
The New York Times, January 30th, 2008: Lilly Considers $1 Billion Fine to Settle Case.

In this example you see that functionality available in the system can be used. Workaround are often also defined because the system missing some functionality and tells you how to use parts of other functions to get your job done.

Most of us know that when defects are found, workarounds are defined; we also test the work around. Only after a while it is forgotten what the workaround actually was. Therefore we should manage the workarounds we define in systems.
We should ask ourselves each time: do we trust the workaround?

Start without a method

I'm the last person who would say that using methods make no sense. As often before, I advise to use them wisely. I can imagine that without any method it might be hard to reach the goal.

You might compare it with getting from point A to point B. In the beginning we didn't had car navigation systems or even paper maps. We started walking, riding, sailing based on our experience. By looking carefully to our environment we adapted the path to go. We even used tools to help us during our journey.

When some trips were made often, wise people drew a map to explain to people with lesser experience how to walk. Even then it was sometimes hard to end up at point B as you needed the skill to read maps, sometimes reading one map type differed from the other.

Currently we are driving our cars, bikes or even walking based on our navigation systems. A tool tells us what to do and when to do what. We stop thinking how to get from A to B. One benefit here is that we have time to other things like watching the environment better, communicating more, etc.

I think the same is now happening in the field of testing; and happened before. Initially we were just testing, finding bugs by pressing the buttons. After a while all kinds of methods were defined like related to testing ISTQB, TMap, TestFrame, TestGoal, TPI, V2M2 etc. Related to development they drew up the different map types like: Waterfall, DSDM, RUP, Agile, etc. They all should help us to get from A to B. And basically, they can work.

Perhaps I'm wrong, what I see happening now is that those methods are also used as tools as they are telling us how to test and when to test. The only difference is here the purpose. Instead of getting from A to B they are to get more free time to enjoy the environment. I suggest that we keep our eyes open to question the environment, use our skills and experience to get from A to B and adapt our routes when we think it is necessary and not because of a method says so or a method forgot to tell you to do.

A method is a good think when it is used as a guideline. Using it as a tool can be done in different ways, you can use it to help you to define your route, it can also be used as goal. When using it is as a goal the focus lies more on implementation of a method instead of using a method.

Collecting evidence

Sometimes it seems testers are just young children, they also want to collect everything, although the reasons are sometimes different.
I can imagine that you, reader, also store your test data for some reason.

Perhaps you can compare it with collecting soccer cards, football card, baseball cards, stamps or what ever. Collecting those items can make you unique; it can give you some additional value. Perhaps you can get money out of it.
Is this the same with test data? Are you just collecting to tell the manager how good you are; how you managed to have everything under control? Or are you collecting to be used as evidence.

Using it as evidence can only help when it is too late and what sense does it make then? Will it help the customer? Perhaps you should ask yourselves in front what to preserve, for what reasons and for how long.

Imagine we define a lifecycle for test data.
We all know these actions:
- preparing
- using
- collecting
- storing
- preserving

What about: destructing?
The initial actions are mostly within a projects lifetime. Destructing should be done when it is no longer necessary. What would you say to start the destruction phase of one project, just after ending the next? This might trigger us to think about what we need to preserve instead saving everything. And it also minimizes the frustration of not finding old data as it is no longer there.

Moments of disbelieve

Sometimes in a lifetime you have some moments when you think: how is that possible. A situation a dealt with just very occasionally is that in the front yard of my house the sun is shining and in the backyard it is raining.

No, I don't have such a big house with a big yard. It just happened. Similar things Moments of disbelieve sometimes also when a product moves from the test environment to production. And in production suddenly it started to rain.

Of course I could have spent time to get the answer why the sun was shining on the other side of my house and it was raining at the backyard. To me it seems some waste of time.

Perhaps we also should deal with certain situations when it happens on productive systems. Instead of searching why it happened, it might be useful to solve the water damage and continue working on the next assignment. Don't let the moment of disbelieve spoil up your resources.

Only when you believe it is not an occasionally thing, you might consider precautions.

When methods can work

There are methods for almost everything you can think of, also in the field of software testing. What often is forgotten is that a method is just a representation and simplification of reality. The danger I see is using methods without questioning that method. Without knowing and checking the boundaries of a method you make unwillingly the assumption that the situation you are in is exactly as the method represents.

I hope you agree that nothing can be identically as described in the method you are tending to use. If you think so you have not only to search for the boundaries for the method you are trying to use and implement, you also have to investigate your own situation much better.

Often I see that people are familiar with methods they have been taught. Over the years they implemented and relied on them, they created some best practices for themselves. In the beginning they asked other people if they understand it correct and used it right. At that moment they were questioning themselves, the method and the environment they were into. After a while it became just a trick they performed and implemented methods the easy way, not questioning anything. I see this as a risk using methods you know. You are no longer fitting the method to the organisation; you are trying to change the organisation to fit the method so you can be successful.

At school I bothered the teachers with asking questions about the method, claiming that those are not useful. Looking back I questioned the method to understand how it works and where the boundaries are. This made me help to implement and use it when possible and triggered me when I tending to cross those borders when implementing them.

It is easy to say that a certain method is not useful; it is harder to tell what parts can be useful and what parts can be of advantage for you.

I suggest that instead of starting with methods or best practices, first start asking questions to get a picture of an organization/ project. Based on this you might be able to create a model for your own of that organization/project. Now you have two models, one model of the organisation and that one called method. Combining this with the known boundaries of the method you might be able to successfully use the method or parts of the method and become successful yourselves.

Different metrics

One of my hobbies is playing with MS Excel. I do this at home, at my assignment and in the past I learned it on Experts-Exchange.com.

The power of MS Excel is collecting data and transform it into information. Initially I tried to create a dashboard which I can use in every project. Unfortunately this was barely possible. Each time, the challenge was to translate the provided data in such a way it represents the same overview of tables and charts. Also requests for information from management differed each time.

I noticed there is a difference between information I need to control the test process and information management need to get informed.

As a project is dynamic and also data is dynamic during the process I start always easy and make the information I can provide also dynamic.

Information for me would be:
- # issues found
- # issues found in combination with system component
- # number test cases /issues open for test
- # test cases /issues in combination with location (what is the status and who should act
- percentage ready (when possible in relation with time left)

Often the difficulty is that information I need is stored in other applications. When possible I download them. Disadvantage can be here if the structure of information is changing or objects are changing in names it become hard to compare. Therefore I start always easy and let the dashboard grow when necessary, or reduce when possible.

Personally I like tables with data, because it enables you to discuss how to understand the information. This is often a complex process, not only you have to explain what information is there, you also have to explain that the data which is shown is correct. Often you can draw several different conclusions out of those tables.

The pitfall here is that this is sometimes hard to explain to management because the like charts. The risk of charts is the discussion behind cannot be shown, information might be missing, people can be mislead. In certain cases I like to keep it simple and fancy.

Here some examples I used over the years:

- the progress meter, can be used per process and for a total. For detailed explanation see: Peltier's Speedometer

Another chart I use is a traffic light to show how the testers feel about the quality, this is often supported by written remarks. Although it is not objective, you still show the feeling and possible threats.

Another daily chart I often use is to see the location of issues. This is a chart which changes on daily basis which I use in combination with available time

A chart managers often like is a chart planned vs actual. This is often hard to get as the data for planned numbers of test cases depends on the availability of testers and also if they are dedicated available for testing. Another remark can be the origin of planned figures. As planned is different then intended.

One of the powerful functions in MS Excel to obtain data unstructured sources is SUMPRODUCT. This one I use often above pivot table, as they avoid growth of workbooks with MB's. Using them wise full, it also enables you to change when data sources are changing.

My advice is start simple and avoid using standard templates as each project will be different. It cost more time to change a template based on provided data and information then start from scratch. Working this way you avoid that unnecessary data has to be maintained and management is informed with non-required information.