BVA test case generator

Recently I read a post about a online tool which can generate test cases based on boundary value (BVA). After a reader of my blog also attended me on this tool I made the free subscription.

On TestersDesk.com you have several tools which I think can be useful like:
- Pairwise TestCase Generator
- Boundary Value TestCase Generator
- N-Way/Random TestCase Generator
- and some more

For example the BVA++ generator is very easy in use, although it doesn’t cover the boundaries we don't know it can be useful .

If you know the values you want to test it is just minutes work to generate the list of cases you need. (within seconds).

I believe this tool can be useful if you have less time available and there is less risk in the functions you are testing. You can use it for:
- generate fast a list of cases you need to cover
- calculate how many time you need to perform those cases. If the generator gives you about 25 cases and you assume that each case will take you about 1/2 hour to execute, you at least know that you might need about 12,5 hour for execution. It can help you to identify in a few minutes how many time you might need to test a function
- the generated cases can also be used to check whether the already defined cases are sufficient.

One of the items I missing here is that you cannot get a list of the total cases so you are able to pick from the total list (if it is there I have missed it.)

Why test management tools can cause pain

I'm aware of management tools who claim to support the testing processes. Some of them can be very useful because they can provide information about the number of defects found, number of test cases created and executed, coverage of those cases related to requirements, and more.

This morning I asked myself why I start projects most of the times using MS Excel. In most of the cases it is because of the lack of test management tools. On the internet you might find all kinds of whitepapers why we should use these tools. It can be easy to convince people why to use it. It might be harder to explain people when to use it. It might be even harder to make people think why not to use it.

Here some points which came up in my mind why it wouldn't be wise to use tools:
- When management didn't decide yet what information they need to make decisions;
- When projects are successful because they have the flexibility to adapt their development process and test process based on their situations: multiple test approaches are used;
- When it takes more time to convince users to use the tool and actually using it instead of working (i.e. testing);
- When people start trusting on the information provided by the tool instead of using common sense;
- When the tool provide too much information which distracts people for the real goal of the project;
- When the tool is used to punish people instead of measuring progress and risks.

I think it is important when certain tools are used they are not dominating your test process and forcing you to make decisions how to act because it won't fit in the tool. Tools shouldn't dictate, it should guide and help. They also should leave space for creativity. Management should be aware to judge the information provided by those tools any time.

TestData Generator

A colleague tester pointed me recently about the importance of test data management and the recently offered site towards test data management: Testdata Management.

Behind one of his links there was a free test data generator: GenerateData.com
GenerateData.com offering a free test data generation tool with several functions to generate "standard" test data in several formats: HTML, MS Excel, XML, CSV and SQL.

Next to generate data online you have to ability to download the script which is under GNU open Source License.

I have to admit that this tool is easy to use although I think you have still to manipulate the data for your own use.

This triggers me to search for more opensource generation tools related to testing. A new world opened in front of my eyes, :)

Here some other open source tools:
dbmonster : can be used for DB tuning
benerator: performing load and performance tests
Database populator : Examines the table and populate it according to it (note: downloadable version is from 2004)

These seems somehow promising, perhaps in the nearby future I can try them sometimes.

If tools are the answer for everything

Often managers are claiming they want to automate the testing process or the test execution.
Lately on seminars or conferences tool suppliers are showing their solution to all kind of answers.
Sometimes the testers are able to convince that using tools can only be fruitful when the process is in place.
Occasionally I wonder does the cost pays it benefits.

Are tools indeed the answer for everything?
When I started working in the software testing business, there were only a handful of test tools available. In certain situations the actually worked. In most cases they became shelf-ware.

If you start searching for tools to support software testing now, then you need more hands to count them. This raises a question to me: Is there no tool which can help us out? Normally I would say: it depends. In this case I'm more tending to the statement: If there are so many tools available, then there is possible a market for it. And if there is a market for it, it has to be based on the principle that "My tool is better then yours". Only other tool supplier would tell the same about you own tool. Based on the number, the chance you find a tool fits your needs is minimized. As by the number the chance on distraction will increase. This distraction might result in a wrong tool selection; which results in creating shelf-ware. Or results in uncontrolled tool implementation.

There is no way back
Perhaps I already mentioned this statement before, only I keep asking this question to myself every time the issue for automation is asked. A former teacher told me that "once you start automating there is no way back."

If you select a tool and start using it; you should have defined a plan for it about the ROI's. If you managed to embed the tool successful in you organization for that moment. You get limited in your flexibility to improve your processes as they have to fit in your tool choises.
Because of this, the used tools might withholding you to improve further.

An option for the future?
What I would like to see is that tools become more flexible. perhaps more SAAS-likely.
Give the organization the option to choose for:
- which UI to use
- which functions to use
- make the application able to use every function you need.

Possible benefits
As organization don't have the need for all functions in current test tools, the tools can be cheaper. The customer pays only for those functions he actual need. Also the need for functions can be incorporated in a improvement plan. this helps also to plan the ROI over the years. more important, the customer can decide what the life-time is for those certain functions and if the process of testing is changing, they are able to use the same tools with different functions.

Perhaps tool vendors should answer the question how they are able to support benefits in the future instead the current need. How they are working on supporting an organization not only to use their tools; also other solutions. As Tools should give an answer to a need and support an organization with obtaining their goals, not only the current processes.

Now: Tool Vendors: Support us with tools able to give us the ability to choose for functionality whether it is your functionality or others to make us succesful using our solution.

I'm not a developer, thought this might help us testers

In the past I ran into an article about visualization of software evolution. It took me some time to dig it up again on the internet.

On this site CVSscan: Visualization of Code Evolution the author ir. Lucian Voinea PDEng gives 3 example tools how to visualize the evolution of code were he claims it can be used as "tool aimed at developers involved in maintenance projects".

Another paper of the same writer on this topic can be found here: Visual assessment of software evolution by L. Voinea, J.J. Lukkien, A. Telea

On his site he explains and offers the following tools:

• CVSscan: Visualization of Code Evolution (file level)
• CVSgrab: Visualization of Code Evolution (project level)
• CSV: Code Structure Visualization
• DreamCode: C++ Abstract Syntax Tree Visualization

You perhaps also want to read the paper he made available from that site called: CVSscan: Visualization of Code Evolution

I think there is also another opportunity here. I think it can be used in Agile/SCRUM projects also. Were writing code is based on usage for that iteration.

Assume that code/functionality changes or better evolves during the project. And as tester you don't immediately know were to look at. Based on turbulence in evolution of code you might decide to test it with more or lesser focus.

Or assume that you first paint a picture with those tools and then measure if the evolution matched the effort you spend on defining Unit tests or other automated tests.

Though I'm quite new on this topic of testing, perhaps others see some advantages of such tools for us testers. Or have already experience with these tools.