What do you do when a person ask you a question? Do you try to answer it immediately or ask an own question to get more information about the purpose?
What do you do when a person explains a problem? Do you try to solve it or ask form more information?
It is in my nature to help people. If a person asks me the time I look at my watch and tells him what is on my watch. If I forgot my watch I look around to check if I see a clock. If no means are available to tell the exact time I try to estimate the time. It is so simple and obvious to respond automatically to triggers we have done before using experience and knowledge we gained before.
The question is: did this help the person? When someone asks for the time and you give the time it seems that the question is answered. Still there might be more behind that question. Perhaps that person used that question to make contact. If you give the time, the chance for further contact is minimized. If that person needed the time to check whether he is late only he is not able to calculate then the question is obviously answered only the person cannot do anything with it. Perhaps he actually wanted to ask if he is late for a certain appointment related to the current time.
The same might happen in testing, when a person tells you that his products is lacking quality you can tell him to test better. Nowadays it normal that in the same sentence we tell them we are able to do that part of testing and inform him that we are able to use methods for this like TMap, ISEB/ISTQB, TestGoal, CDT, SmarTest or whatever. Do to our experience it became our nurture to tell about our skills and experience.
Telling about skills and experience won't help a person. Using them might help. Only it can only help if you know it will fit the problem/ question. Before you start using your skills/experience about methods you first need to check if it is sufficient. It is important to start gaining information first. Then you can decide which solution/method is the best for helping the person.
I think here the tricky part comes: It is in our nurture to start asking questions based on our knowledge and experience. If that knowledge is restricted then the gained information is also restricted. Perhaps it is useful for the method you like to use; the pitfall here is that you might be forced to change customer’s processes and make them accept the disadvantages to make your method work.
I don't know exactly how to resolve this problem. At least I believe sticking to the knowledge/rules of one method is insufficient. Also the knowledge of 2 similar methods like ISTQB and TMap won't work. You will be forced thinking in a certain pattern although this pattern cannot be the answer. It might help you to lead to a solution. Currently I'm reading more beyond these approaches, listening to ideas of others and trying to make up my own ideas. I'm still aware I'm at the beginning of the journey where it helps to start with the statement: Currently no method is useful, let us first see what want to do and what we are currently doing.
Saturday, June 6, 2009
Blocking your nurture
Posted by
Jeroen Rosink
at
9:02 AM
0
comments
Labels: Ideas, Metaphor, Testing in General
Do you trust the Work-Around
In the past I already made a post related to workarounds March 23rd, 2008: Start managing your Workaround's!
In that posting I advice to start with documenting workarounds and managing them based on their lifecycle. We should avoid that workarounds get normal desired functionality by just using it; the next step would be that that usage become normal expected activities.
An article I read a while ago is related to functionality which is default in MS Outlook, auto complete e-mail addresses, only that functionality is often not embedded in processes. It is there, only organizations often don't tell you to use it or not.
CNet new: February 5th, 2008: Lilly's $1 Billion E-Mailstrom;
The New York Times, January 30th, 2008: Lilly Considers $1 Billion Fine to Settle Case.
In this example you see that functionality available in the system can be used. Workaround are often also defined because the system missing some functionality and tells you how to use parts of other functions to get your job done.
Most of us know that when defects are found, workarounds are defined; we also test the work around. Only after a while it is forgotten what the workaround actually was. Therefore we should manage the workarounds we define in systems.
We should ask ourselves each time: do we trust the workaround?
Posted by
Jeroen Rosink
at
5:36 AM
0
comments
Labels: development, Testing in General
Friday, June 5, 2009
Start without a method
I'm the last person who would say that using methods make no sense. As often before, I advise to use them wisely. I can imagine that without any method it might be hard to reach the goal.
You might compare it with getting from point A to point B. In the beginning we didn't had car navigation systems or even paper maps. We started walking, riding, sailing based on our experience. By looking carefully to our environment we adapted the path to go. We even used tools to help us during our journey.
When some trips were made often, wise people drew a map to explain to people with lesser experience how to walk. Even then it was sometimes hard to end up at point B as you needed the skill to read maps, sometimes reading one map type differed from the other.
Currently we are driving our cars, bikes or even walking based on our navigation systems. A tool tells us what to do and when to do what. We stop thinking how to get from A to B. One benefit here is that we have time to other things like watching the environment better, communicating more, etc.
I think the same is now happening in the field of testing; and happened before. Initially we were just testing, finding bugs by pressing the buttons. After a while all kinds of methods were defined like related to testing ISTQB, TMap, TestFrame, TestGoal, TPI, V2M2 etc. Related to development they drew up the different map types like: Waterfall, DSDM, RUP, Agile, etc. They all should help us to get from A to B. And basically, they can work.
Perhaps I'm wrong, what I see happening now is that those methods are also used as tools as they are telling us how to test and when to test. The only difference is here the purpose. Instead of getting from A to B they are to get more free time to enjoy the environment. I suggest that we keep our eyes open to question the environment, use our skills and experience to get from A to B and adapt our routes when we think it is necessary and not because of a method says so or a method forgot to tell you to do.
A method is a good think when it is used as a guideline. Using it as a tool can be done in different ways, you can use it to help you to define your route, it can also be used as goal. When using it is as a goal the focus lies more on implementation of a method instead of using a method.
Posted by
Jeroen Rosink
at
4:35 AM
0
comments
Thursday, June 4, 2009
Collecting evidence
Sometimes it seems testers are just young children, they also want to collect everything, although the reasons are sometimes different.
I can imagine that you, reader, also store your test data for some reason.
Perhaps you can compare it with collecting soccer cards, football card, baseball cards, stamps or what ever. Collecting those items can make you unique; it can give you some additional value. Perhaps you can get money out of it.
Is this the same with test data? Are you just collecting to tell the manager how good you are; how you managed to have everything under control? Or are you collecting to be used as evidence.
Using it as evidence can only help when it is too late and what sense does it make then? Will it help the customer? Perhaps you should ask yourselves in front what to preserve, for what reasons and for how long.
Imagine we define a lifecycle for test data.
We all know these actions:
- preparing
- using
- collecting
- storing
- preserving
What about: destructing?
The initial actions are mostly within a projects lifetime. Destructing should be done when it is no longer necessary. What would you say to start the destruction phase of one project, just after ending the next? This might trigger us to think about what we need to preserve instead saving everything. And it also minimizes the frustration of not finding old data as it is no longer there.
Posted by
Jeroen Rosink
at
4:57 AM
0
comments
Labels: Metaphor
Wednesday, June 3, 2009
Moments of disbelieve
Sometimes in a lifetime you have some moments when you think: how is that possible. A situation a dealt with just very occasionally is that in the front yard of my house the sun is shining and in the backyard it is raining.
No, I don't have such a big house with a big yard. It just happened. Similar things Moments of disbelieve sometimes also when a product moves from the test environment to production. And in production suddenly it started to rain.
Of course I could have spent time to get the answer why the sun was shining on the other side of my house and it was raining at the backyard. To me it seems some waste of time.
Perhaps we also should deal with certain situations when it happens on productive systems. Instead of searching why it happened, it might be useful to solve the water damage and continue working on the next assignment. Don't let the moment of disbelieve spoil up your resources.
Only when you believe it is not an occasionally thing, you might consider precautions.
Posted by
Jeroen Rosink
at
5:48 AM
0
comments
Labels: Metaphor
Tuesday, June 2, 2009
When methods can work
There are methods for almost everything you can think of, also in the field of software testing. What often is forgotten is that a method is just a representation and simplification of reality. The danger I see is using methods without questioning that method. Without knowing and checking the boundaries of a method you make unwillingly the assumption that the situation you are in is exactly as the method represents.
I hope you agree that nothing can be identically as described in the method you are tending to use. If you think so you have not only to search for the boundaries for the method you are trying to use and implement, you also have to investigate your own situation much better.
Often I see that people are familiar with methods they have been taught. Over the years they implemented and relied on them, they created some best practices for themselves. In the beginning they asked other people if they understand it correct and used it right. At that moment they were questioning themselves, the method and the environment they were into. After a while it became just a trick they performed and implemented methods the easy way, not questioning anything. I see this as a risk using methods you know. You are no longer fitting the method to the organisation; you are trying to change the organisation to fit the method so you can be successful.
At school I bothered the teachers with asking questions about the method, claiming that those are not useful. Looking back I questioned the method to understand how it works and where the boundaries are. This made me help to implement and use it when possible and triggered me when I tending to cross those borders when implementing them.
It is easy to say that a certain method is not useful; it is harder to tell what parts can be useful and what parts can be of advantage for you.
I suggest that instead of starting with methods or best practices, first start asking questions to get a picture of an organization/ project. Based on this you might be able to create a model for your own of that organization/project. Now you have two models, one model of the organisation and that one called method. Combining this with the known boundaries of the method you might be able to successfully use the method or parts of the method and become successful yourselves.
Posted by
Jeroen Rosink
at
5:53 AM
0
comments
Labels: Ideas, Test Methods
Monday, June 1, 2009
Different metrics
The power of MS Excel is collecting data and transform it into information. Initially I tried to create a dashboard which I can use in every project. Unfortunately this was barely possible. Each time, the challenge was to translate the provided data in such a way it represents the same overview of tables and charts. Also requests for information from management differed each time.
I noticed there is a difference between information I need to control the test process and information management need to get informed.
As a project is dynamic and also data is dynamic during the process I start always easy and make the information I can provide also dynamic.
Information for me would be:
- # issues found
- # issues found in combination with system component
- # number test cases /issues open for test
- # test cases /issues in combination with location (what is the status and who should act
- percentage ready (when possible in relation with time left)
Often the difficulty is that information I need is stored in other applications. When possible I download them. Disadvantage can be here if the structure of information is changing or objects are changing in names it become hard to compare. Therefore I start always easy and let the dashboard grow when necessary, or reduce when possible.
A chart managers often like is a chart planned vs actual. This is often hard to get as the data for planned numbers of test cases depends on the availability of testers and also if they are dedicated available for testing. Another remark can be the origin of planned figures. As planned is different then intended.
Posted by
Jeroen Rosink
at
7:00 AM
2
comments
Labels: Metrics
Sunday, May 31, 2009
BVA test case generator
Recently I read a post about a online tool which can generate test cases based on boundary value (BVA). After a reader of my blog also attended me on this tool I made the free subscription.
On TestersDesk.com you have several tools which I think can be useful like:
- Pairwise TestCase Generator
- Boundary Value TestCase Generator
- N-Way/Random TestCase Generator
- and some more
For example the BVA++ generator is very easy in use, although it doesn’t cover the boundaries we don't know it can be useful .
If you know the values you want to test it is just minutes work to generate the list of cases you need. (within seconds).
I believe this tool can be useful if you have less time available and there is less risk in the functions you are testing. You can use it for:
- generate fast a list of cases you need to cover
- calculate how many time you need to perform those cases. If the generator gives you about 25 cases and you assume that each case will take you about 1/2 hour to execute, you at least know that you might need about 12,5 hour for execution. It can help you to identify in a few minutes how many time you might need to test a function
- the generated cases can also be used to check whether the already defined cases are sufficient.
One of the items I missing here is that you cannot get a list of the total cases so you are able to pick from the total list (if it is there I have missed it.)
Posted by
Jeroen Rosink
at
9:39 AM
0
comments
Labels: Tools
How many documents can you create?
Often I hear the saying a good tester is a lazy tester. This doesn't mean that testers don't want to do nothing; it is just that testers just want to do just that what is necessary. Like why execute all 1000 test cases if based on test techniques these can be minimized to a number of 52?
Over the years I noticed that to control projects a lot of documentations is created. This made me curious about which types of documents I had to deal with or might have created if I follow the methods like PRINCE2, ISTQB, TMap or others.
Here a list of documents I had to face with:
- Project assignment document
- Initial test advice report (before actual testing starts)
- Master test plan
- Risk analysis report
- Product Risk Analysis Template/ checklist
- Product Risk Analysis report
- Detail Test Plan (for each phase one: SIT, FAT, UAT, PAT)
- Test strategy checklist
- Test strategy document (finally it would become part of the MTP and DTP)
- Weekly progress report
- Daily progress update report
- Daily progress templates (to gain information from for the daily progress reports)
- Deviations report
- Phase end report
- Escalation reports
- Logical test specification document
- Physical test specification document
- Test Script
- Test process dashboard
- Intake test basis checklist
- Checklist intake test tools
- Checklist test process evaluation
- Test Process Evaluation report
- Test Process Improvement Checklist
- Test process improvement report
- Checklist test techniques
- Test schedules
- Test estimation document
- Defect registration
- Defect screen print templates
- Test Execution Quick reference cards
- e-mails
- Review template
- Review report
- .....
Of course there are some more documents I had to face with. Looking at this list I just wonder: Is there still some time left after generating these documents to actually perform tests? Of course these can be all useful documents. We should continue asking the systems and the processes questions and preserving the outcome of it. Only: are we working for those documents or are those documents work for us?
Posted by
Jeroen Rosink
at
7:13 AM
0
comments
Labels: Test Methods, Testing in General
Saturday, May 30, 2009
Why test management tools can cause pain
I'm aware of management tools who claim to support the testing processes. Some of them can be very useful because they can provide information about the number of defects found, number of test cases created and executed, coverage of those cases related to requirements, and more.
This morning I asked myself why I start projects most of the times using MS Excel. In most of the cases it is because of the lack of test management tools. On the internet you might find all kinds of whitepapers why we should use these tools. It can be easy to convince people why to use it. It might be harder to explain people when to use it. It might be even harder to make people think why not to use it.
Here some points which came up in my mind why it wouldn't be wise to use tools:
- When management didn't decide yet what information they need to make decisions;
- When projects are successful because they have the flexibility to adapt their development process and test process based on their situations: multiple test approaches are used;
- When it takes more time to convince users to use the tool and actually using it instead of working (i.e. testing);
- When people start trusting on the information provided by the tool instead of using common sense;
- When the tool provide too much information which distracts people for the real goal of the project;
- When the tool is used to punish people instead of measuring progress and risks.
I think it is important when certain tools are used they are not dominating your test process and forcing you to make decisions how to act because it won't fit in the tool. Tools shouldn't dictate, it should guide and help. They also should leave space for creativity. Management should be aware to judge the information provided by those tools any time.
Posted by
Jeroen Rosink
at
8:55 AM
0
comments
Labels: Ideas, Test Management, Tools
