If tools are the answer for everything

Often managers are claiming they want to automate the testing process or the test execution.
Lately on seminars or conferences tool suppliers are showing their solution to all kind of answers.
Sometimes the testers are able to convince that using tools can only be fruitful when the process is in place.
Occasionally I wonder does the cost pays it benefits.

Are tools indeed the answer for everything?
When I started working in the software testing business, there were only a handful of test tools available. In certain situations the actually worked. In most cases they became shelf-ware.

If you start searching for tools to support software testing now, then you need more hands to count them. This raises a question to me: Is there no tool which can help us out? Normally I would say: it depends. In this case I'm more tending to the statement: If there are so many tools available, then there is possible a market for it. And if there is a market for it, it has to be based on the principle that "My tool is better then yours". Only other tool supplier would tell the same about you own tool. Based on the number, the chance you find a tool fits your needs is minimized. As by the number the chance on distraction will increase. This distraction might result in a wrong tool selection; which results in creating shelf-ware. Or results in uncontrolled tool implementation.

There is no way back
Perhaps I already mentioned this statement before, only I keep asking this question to myself every time the issue for automation is asked. A former teacher told me that "once you start automating there is no way back."

If you select a tool and start using it; you should have defined a plan for it about the ROI's. If you managed to embed the tool successful in you organization for that moment. You get limited in your flexibility to improve your processes as they have to fit in your tool choises.
Because of this, the used tools might withholding you to improve further.

An option for the future?
What I would like to see is that tools become more flexible. perhaps more SAAS-likely.
Give the organization the option to choose for:
- which UI to use
- which functions to use
- make the application able to use every function you need.

Possible benefits
As organization don't have the need for all functions in current test tools, the tools can be cheaper. The customer pays only for those functions he actual need. Also the need for functions can be incorporated in a improvement plan. this helps also to plan the ROI over the years. more important, the customer can decide what the life-time is for those certain functions and if the process of testing is changing, they are able to use the same tools with different functions.

Perhaps tool vendors should answer the question how they are able to support benefits in the future instead the current need. How they are working on supporting an organization not only to use their tools; also other solutions. As Tools should give an answer to a need and support an organization with obtaining their goals, not only the current processes.

Now: Tool Vendors: Support us with tools able to give us the ability to choose for functionality whether it is your functionality or others to make us succesful using our solution.

Ignore priorities: Transport Driven Testing

I think we are used to work with priorities related to what to test or what not to test. Based on priorities we also demand deliverance of functionalities by development.
Business priorities are also used to schedule which functionalities should be delivered by development first to make sure that those items are delivered first which gives the business optimal gains.

Sometimes also functionality is delivered because it was easy to fix.

In SAP you use the term "Transports" instead of terms like: "deployment", "delivery", "shipment", etc

To control and schedule transports in SAP you can use SAP Solution Manager. A benefit here is that based on a status you are able import your transports from the test environment to the production environment.

To build a solution in a transport you have to attach them to Solution Manager tickets. A ticket can contain one or more transport numbers. With these transports you made modifications to SAP objects. Each newer ticket might contain modifications on the same objects. a newer version of those changed objects exists now in the library.

Here comes the difficulty:
Imagine that ticket "A" contains a small modification which had lower business priority only it was quick to fix. And ticket "E" contains a high priority solution.

What if there is no tester available to test a ticket called "A" and you are already testing a ticket called "E" which came in later in the transport list and contains modifications on the same objects? If you confirmed ticket E is correct and it will be shipped using an emergency transport to production and later on you import the ticket "A", then the newer solution of objects are overwritten by the older version. This might result in a situation that the solution in ticket "E" doesn't apply anymore.

To prevent this to happen you have to force the tester also to test ticket "A" before an import of objects is made. You are now in a situation where imports are no longer based on business need; it is based on transport need. In this case a ticket is defining the priority for testing. With other words: the location of a transport in the transport list defines which ticket has the highest priority for testing and no longer the business need.

In SAP Solution Manager you have the ability to use the Import All List to monitor which transports should be tested first. Based on the status of the ticket like "Tested, OK" you can define which transports can be imported without major risks. If there are transports which don't have this status, you can place a STOPMARK before that transport. All transports after that STOPMARK have to be imported on manual request. This introduce the risk that when during the next test phase those transports are desired by business and have to be imported using an emergency transport you might have to import other objects as well.

In this situation, the release content is based on business priority, you have to make sure you don't get the easy things also in which are easy to fix and hard to get a tester for. When the release content is fixed, you have to test not only on complexity and business gain; you also have to test to make sure that the STOPMARK is set as low as it can be. This means that you have to work according FIFO (First in-First out), first import the objects which are entered first on the list as the next one might contain a newer version of the object.
This means you have to force the test execution based on the transport list, so you might call it transport driven testing.