Saturday, August 9, 2008

Development Models and the impact on Testing

During my career as tester I learned that development methods have impact on testing. Also when sometimes a tester is working based on a test method it has impact on the success of the development method.

In books related to software testing often an approach is defined which is based no the Waterfall method. In the newer books they adjust that method to make it fit for incremental and iterative methods. Sometime this is done by only mentioning that it also works for other approaches.

In this Blog I'm trying to give some information about 3 development streams. It might not be completely perfect but I think it might be useful for the readers to give a picture how I see things. While I already know I want to write, I also know that this article will be much longer then I normally write. To miss information I decided not to split this topic in several other postings. Therefore I will give you some table of content:
- 3 approaches of development
- How development methods are linked to these models
- Short overview benefits and disadvantages per development method
- Overview how other models are related to these methods
- Impact towards software testing.

3 approaches of development
In general you have 3 development approaches. Waterfall, Incremental and Iterative (Evolutionary).
For more information you might check out the links which refer to description on Wikipedia.

In short these approaches can be described as following:
Waterfall approach: The whole system will be implemented at once;
Incremental approach: The system is divide is several steps, and is implemented in that order. Once a step is implemented you can not go back to a previous step to change functionality or do some "refactoring";
Iterative approach (Evolutionary): This is an approach were you continue development towards a final system. You are able to go back and change the code of the previous iteration. every iteration delivers a better system, even without adding new functionality.



How development methods are linked to these models
Often several types of development are called to be a variant on Agile Development, I rather prefer to keep speaking in terms of approaches. In this case Agile belongs to the more Evolutionary approach. Sometimes an approach belongs based on the actual implementation to either Incremental or Iterative. In that case you should not only look at the name of the development method like RUP or DSDM. You also have to investigate how that method is implemented. Perhaps the following picture gives some overview.


Short overview benefits and disadvantages per development method
In the last few years we tending to neglect Waterfall-approaches and claiming that methods as RUP, XP and Agile are always better. Only in some situations it is better to play save then take the risk of using methods you don't control on a level which is necessary. Mostly systems which has a long life-time like aerospace systems, life care systems or systems where you are a strong market leader and those don't change that much. I could be wise to use for these systems a waterfall approach. In the figures below an overview is given about the benefits and disadvantages of the 3 approaches. I know they are not complete. Still it can be handled as a guideline.







Overview how other models are related to these methods
To understand the impact of these approaches on Software Testing you have to understand how an approach effects the organization. To get a short overview of this I took several other models and views which can represent an organization.
Models:
- Organizational Model: Documentation about the organizational processes, procedures and activities;
- Information Model: Documentation how the organizational processes are supported/executed;
- Data Model: Documentation about the kind and location of information used;
- Process Model: Documentation how data is processed to support the pervious models .

Views:
- Conceptual/Logical View: High level view of the organization;
- Physical/Technical View: View of the architectural and informational representation of the system;
- Implementation View: How the system should be implemented in the organization.

In the tables below I tried to translate the development approaches towards the impact on documentation using several other models and views. To visualize this I used several colors.

When a certain cell is colored Green, then during the project this will not change much after it is finished. When the color is Yellow, there is a small chance it will change. The color Orange represents an obvious chance of change and the color Red: there will be changes.

The situation here is that when a chance occurs/ is made. The "defined" steady objects might also change when it is a result of new awareness. When this happens the test cases will change because they are built based on this documentation. Because changing documentation in previous steps takes time it gives an impression how steady the development process will be and therefore the test process. On the other hand it also give a feeling for ability to adapt your test process on those changes.

Views for a Waterfall-approach


Views for a Incremental-approach

Views for a Iterative-approach



Impact towards software testing.
Below you find a short overview of the characteristics of the development approach, their impact on the project development process and the impact on the test process. Keep in mind that these items are not complete. It is just mentioned to serve as a guideline. There is always a dependency towards the actual implementation of the development method.



I hope this blog gives you more feeling how development processes are impacting the test process. Bare with me when I made some miss interpretations. It came never to my mind to make a statement about which approach is better then the other. It should help you as a guideline to identify impact on your own process and therefore helping to identifying risks and controlling those.

Sunday, August 3, 2008

Request For Knowledge (RFK)

Terms like RFC or CR are well known in the development processes. They often are raised during the testing phase and some new features have to be developed. Most of the time this has its impact on the testing process. It is hard to test them within the same time period of the testing phase.

Mainly these are created based on new knowledge of the capabilities of the technology or system or even process. Prevention of this can be done by performing formal reviews or inspections. The outcome is registered in some way. Though these are not directly linked in testing strategies were additional risk might be involved.

For example: If formal reviews are executed, errors are identified and registered in a review log. The information about how many defects in documentation were found in certain areas are not used as they already solved those issues. So the risk for errors in the system is reduced. Only the change for RFC's is not implicitly reduced.

Perhaps we should introduce the term Request For Knowledge (RFK) and also make a formal process of it.

If I have to think about a definition for this it could be something like this:
A RFK is a formal request for additional information about a system, architecture, process, data or requirement.

The process could be something like this:
1. Create and maintain a RFK backlog; every person of the project can add items to it.
2. Assign a RFK backlog owner;
3. Start monitoring during requirement-specification phase
4. Every RFK should have a status, owner and result
5. For every phase transition, also handover the RFK-backlog.
6. Before the handover, identify the impact of outstanding RFK's on the next phase
7. Take counter actions for that impact.

The reason to do this is creating a process to get more and new information which could help reduce RFC's which have to be delivered tight away during testing phase.
I think when we start monitoring every area / subsystem/ process where questions are raised can give us information where functionality might change.

If in a period of time certain areas keeps requesting for knowledge then the chance is huge that the functionality doesn't meet the actual desired requirements. So the chance is also huge RFC's are introduced on a later stage.
On the other hand, if certain areas don’t have RFK's, perhaps this is a signal that the requirements are also not defined thoroughly.

If you prioritize the areas and combine it with number of RFK's you are able to identify risk areas earlier.



If we assume that based on above table "Process A" had a number of RFK's for "Sub System 2" which was unexpected high. And we see that "Process A" is also used in "Sub System 1". The impact of changes in "Sub System 2" when information is retrieved during testing phase brings high potential risk for "Sub System 1". If we have this information in front of starting testing, the test strategy can be adopted to cover this risk.

These figures can also be used identify business involvement. In projects sometimes certain department give full involvement right from the beginning. Certain departments start involving at the end of the project and the chance of raising RFC's is increased.
These figures can also be used if certain area's have a lot of RFK's, the users are instructed comprehensively by more detailed manuals and training as the chance they miss-use the system is increasing.

Perhaps you, reader, can come up with other usages of the RFK's.

Sunday, July 27, 2008

Software testing and Astronomy

I think watching the stars can be compared to software testing. If you don't have any idea were to look at, if you don't have any jargon to use it just seems to be nice beautiful stars. The same is with systems, if you don't know were to look an application is just an application.

To proof or confirm quality of software applications organizations performs software test. To do this on a professional level, they acknowledge the need of software testers. Software testing is a profession and a science. So is studying the stars also: Astronomy.

Though it seems that these 2 sciences are the opposite: according to article on Wikipedia: Astronomy is one of the oldest sciences; software testing might belong to one of the newest sciences.

Imagine you sitting on a chair outside and the sky is clear and you look at the stars. To get a picture how the sky is you first try to get some reference point. I normaly start searching for the Great Bear.

The Great Bear is a pattern I well know and is from my point of view easy to recognize as there are not much stars to blur my vision. From that point I continue my exploration of the sky. As I know there are other patterns of stars I try to find them, though I'm not sure if I link the correct stars. Sometimes I got distracted by other beautiful things in the sky like: falling stars, satelites, planes or those rarely found UFO's.

Unfortunatly I'm not able to continue my exploration till the end as the Universum is too big, my time ends up as sometimes you need some sleep and of clourse from my chair I have only limited view of the Universum.

According to me: Astronomy can be compared to software testing. As there is too much to look at, you never can be complete, time is limited, without good specs and lesser knowledge it is hard to complete your queste though it can still be fun and there are a lots of things which can distract your attention.

To watch the application/system as watching the stars you need:
1. Location were to start: A postition which can be easily identified and which is close to other objects you want to look at;
2. Description/map how that position can be identified: like requirements or a symbol like the Great Bear;
3. Clear notation so no misunderstanding takes place: Some people call it Great Bear, others know the pattern as Big Dipper;
4. Direction were to continue: Continnue left of the Great Bear. In software testing this is part of the strategy;
5. Description of area: what part of the universum are you going to explore. In software testing this can be covered by test objects. Some times the objects are small enough and can be covered in one test case, other times you have multiple test cases;
6. Schedule: how much time and resources are available: Do you watch the sky alone or with others and in which period.
7. Time to start exploring: as watching the universe it mostly make sense to start when it is dark outside not cloudy. Same with software testing: Make sure that you are able to identify your starting point and there are more items to explore.
8. level of detail to watch: you can watch for patterns in the sky, they mostly have names and can be identified based on a map of the stars. Those stars itselve also have names. Are you looking for patterns, single stars or a combination of it?
9. Describe were to look at, make sure you don't get distracted by falling stars, other stars, planes etc. Still it is important that you are able to identify it as a falling star. In software testing it could be a bug in an area you should not focus on. Still it is important to identify that it was not a bug.
10. Be aware you are not able to cover everything, the universum is large, so are applications. Make sure you cover the right things
11. Keep track of what you are doing so you can continue the next night so you don't have to start again
12. If no map is available, perhaps you have other knowledge about the universum. As constellations are picked by humans also, it might be obvious you can combine a number of stars which looks like them. In this case make sure you made it acceptable that there is a risk you picked too much or too less stars. In Testing this can be identified as coverage.
13. geographic position can also help: sitting on a mountain you are closer to the stars, in a space shuttle you are have clearer vision. So is it with testing, looking at code, you have more detailed view of the items then when you look at a complete system.
14. ........

Of course there are more points which are similar to astronomy and software testing. Perhaps you, reader, can come up with more.

I think we have to keep in mind that the science Astronomy is still learning and exploring. We as testers also should no be affraid of learning new things and new approaches.

Monday, July 7, 2008

The power of Three

If you take some time and step aside from your daily activity you might have noticed that there is some kind of structure in the activities you are doing and also in the environment you’re in. If you have enough imagination you might come back every time on the number 3.

In old days of school I learned the words "Trias Politica". Doing some wiki-search learned me that the original thoughts about this come from Charles de Secondat, baron de Montesquieu he described in Montesquieu's tripartite system "division of political power among an executive, a legislature, and a judiciary. He based this model on the British constitutional system, in which he perceived a separation of powers among the monarch, Parliament, and the courts of law."

Of course you might come up with more examples which extend the number 3. When thinking you can come up with examples like these:
a. test manager - test coordinator - tester
b. time - money - quality
c. system under test - test environment - testware
d. requirement - functionality - test script
e. errors - faults - failures
f. development - testing - implementation
g. plan - do - check
h. junior - medior - senior
i. test preparation - test specification - test execution
j. ....

Looking at these items you will see that you might have to deal with it during your job. If you can only figure 2 items you might spend some time to identify the third item. If you come you come up with more items it might trigger you to separate those until you have those items which are in perspective to each other. To get this you might keep in mind which belongs to these: executive, legislature, judiciary.

I think this is important as these items will influence you daily work. They can support your activities or endanger the things you are doing. I think it is important to know who your “allies” are and who your “foes” are. I think it is important to know what you miss or perhaps were you made a "wrong" linkage to one of the 3 items of the "Trias Policita".

This might help you better to understand your work, your role and your activities. Based on this you can improve your selves and the process.

Of course there are more things to say about this topic. Perhaps you think it doesn't work in your situation. I like to hear about it and learn from you fellow testers.

Saturday, June 14, 2008

Which errors do you accept?

You should test everything! I don't want any errors in production! The system should be a high quality system! And it all should be done within time and budget.

These are some phrases I hear often when it comes to software testing. Only when asking the questions: "What is everything?", "What is a high quality system?", "Within which timeframe you don't want see any errors?" Often the answer is: "Just like I said, just do it!"

Often these are signals people don't exactly know what they want. Still, expectations are in these circumstances high related to testing. They expect you are the solid rock to make sure they didn't miss anything. Only in these conditions it is hard to test.

I think the question asked is wrong. Perhaps it helps asking: "Which errors do you not want to see in production?"
The answer to this question should be translated to risks based on the functionality of the system. And before testing, ask the organization on what level they are willing to accept those risks. This might help to define the test strategy.

Perhaps these answers might be given:
1. "I have not enough knowledge about the errors I don't want to see!"
2. "These specific errors I don't want to see."
3. "Any error is wrong."
4. "You are the expert! You have to tell me."
5. "Don't ask me what is acceptable to go wrong, give me information what goes right!"
6. "Sorry, I don't have the time to answer your question."
7. "Who are you?"
8. "I do not speak Dutch!"
9. ....

If you take a look at these answers you see that in most cases (except answer 2) information is missing. If information is missing how are you able to test properly?

If it is clear what errors are not acceptable, you might consider to proof that those errors are not happen in the environment, instead of testing everything.
If you are in the lead to give information, you might based on functionality and complexity define a strategy where complex functions/modules with high business gain are tested thoroughly and others with lesser effort. And make it visible if it fits the timeframe of the project. Of course it doesn't. You know can ask the organization what risks they accept, what should be tested lesser.

It might help if you make it visible what impact functionality has on the testing process. Perhaps the figure below can help you to explain. I have to mention it is not complete. And off course you have to customize it to make it fit in your situation.


Instead of using Technical Impact, you might think of classifying errors not to be seen on the system. You might use: "Maturity of Identified Errors". Only in this case: If errors can be identied specifically you classify them as high and otherwise as low.

I expect that if the organization is not able to identify not wanted errors, and they see what you are able to test (Error Guessing) they might consider spending some time for identification.

Please don't see this approach as the perfect tool. Use it as a "talking picture"

Perhaps this helps you out.

The impact of an obvious small change

Some days ago I heard a commercial on the radio about the new Mercedes. In this commercial they talked about the wish of the driver that a certain flap should be used less often. To do this Mercedes adapted their car on a certain 18 points. First I thought it was about some very technical requirement the driver had. At the end of the commercial it was all about the flap above the rear tire: The gasoline flap.

For me this was a good example how a requirement can be stated in simple terms only the impact of that wish has huge technical impact to the car.

Somehow I got a déjà-vu moment. How often during testing new requirements are introduced by the business while your almost at the end of testing? Isn’t the business not claiming that it should be very simple to build and still bring the system to production on time?

I don't expect that Mercedes tested their car just by measuring the times the flap was used. Is it not more obvious that they tested all those 18 changes singular and integrated? I think so, because they were able to translate the requirement to the initial problem. It was much easier for them to minimize the usage of that flap by gluing it to the car. Or, build a timer on the flap to prevent opening to often. Or, hide the flap. The initial requirement would be solved this way. The flap can not be used that often or entirely by the driver.

In projects such requirements are often introduced. It could be some additional field on the screen or on a report. And most often the business insists that those fields are build. For them it seems very simple to build. Just drag another field to the screen and the issue is solved. Only they are most of the time not informed about the technical impact and certainly not about the impact for the testing process.

Imagine that a certain field needs other data. Queries have to be rebuilt. Or even worse, a new table has to be introduced. In such situation you don't only look if the field is on the screen and contains some data. You also want to test what other impact has the introduction of this feature.

If organizations have a well defined process, they are writing a Change Request. Based on this CR a technical impact is determined. And if the impact is not that huge decision is made if the risk of failure is acceptable against the business value. What I see is that often the test impact is forgotten in those impact analyses. Because of this missing information the business decide to build the CR.

After the CR is build the testing part starts. Here some points I think what can go wrong. Perhaps you see other points or even experience other points. I am very curious to those.

1. In impact analysis no attention was paid towards the test impact, this might result in delay of the project as testing takes more time then expected. "It was just a small change! Why need so much time for testing?";
2. If impact analysis is only done in relation to that certain function/module and not towards the impact in relation with other main functions/modules, the selection of regression test cases might be wrong. The change of only testing the gasoline flap is bigger then testing those 18 changes in the system. New errors might be introduced;
3. If the technical impact is low, and also the business gain is just medium, or even low. Why bother the change of introduction of new failures. It might help also to consider here the impact on testing;
4. If there is no good documentation about the system. Technical impact is decided based on lines of code. How can you tell what the integrated impact would be? And also important, how are you able to select the proper test cases?;
5. If business gain is high and technical impact is medium/high and time is short, testing is mostly done in a bad, quick manner. How do you know which risks to accept by introducing this feature to production? Under pressure, often a basic test set is chosen. And often this seems to be too less for measuring those accepted risks;
6. If impact analysis is talking about 18 changes, how strong is your development process that there are not somewhere unwritten changes made to make those 18 work? If you don't know about other small changes or code workarounds, how can the tester know about those?;
7. Another situation sometime happen is that business doesn't know about the impact on their processes of those changes. Development doesn't know in front about the technical impact. And still the tester gained the order to test everything thoroughly. Is this an order you can give the tester in these circumstances?;
8. What about the situation when only one test environment is available, you testing a huge impact change and during testing a production failure is introduced to test. This might lead to a situation that the production failure must be delivered with that rarely tested change towards production. Make sure you have multiple test environments or you are in control of your environments.
9....


As mentioned earlier, this list is definitely not complete. I hope this gives you a picture that a small change for business is not obviously a small change on testing. I think it is important for all parties involved not to make assumptions to quickly and keep communicating and respect each others arguments. Perhaps we should stop making decisions based on time pressure. Making decisions based on arguments might be much saver and even cheaper.

Sunday, June 1, 2008

A strategy game: Civilization 2

One of my favorite games to play is Civilization II. And I think it has all to do with software testing. Here a very short explanation to relate this game to software testing.

This game is a strategy game, were you have to build a civilization where you can choose for 3 goals to win the game:
1. Be the first civilization who builds and lounge a spaceship as first;
2. Eliminate all other civilizations;
3. Keep playing until the year 2020.

The reasons why I think it is like software testing are:
1. before starting you have to determine how the world have to look like. Should there be a large landmass (longer time to play) or a smaller area (shorter time to play);
2. how many opponents you have to beat. If you choose for the maximum there are other circumstances you have to take care of in combination of you strategy to win;
3. were do you focus initially on: building lesser cities with all improvements of larger armies of lots of cities with lesser improvements. This decision will influence the rest of the game;
4. do you "cheat" or just continue playing when something did not went as you hoped for?;
5. when "cheating" which outcome do you accept at that moment.

Ad1: translated to testing: do you choose for time boxing or do you test until you met all acceptance criteria.
Ad2: If a testing team is larger you have to communicate more with the team and make decisions every one satisfies. The lesser opponents, the easier it might be to satisfy them.
Ad3: An optimal designed city might be translated to an optimal defined test process. Only this might prevent you to become the first civilization to build and lounge a spaceship. It is very good to make the year 2020. Focusing on a lesser defined process by building as much as armies (perhaps translated to test scripts) might help you conquer the world and become the strongest.
Ad4: cheating can be done by saving the game. And when you get into battle restore the game until you get the desired result. Or when entering a "hidden" village, accept the newly gained knowledge, or perhaps you found another city or a certain amount of money. This can be compared with regression testing: keep retesting until the expected or acceptable result is there. And not continuing with a loss of armies or accepting the horde of barbarians you released.
Ad5: Based on the situation and moment of the game you restore your saved game and accept in the beginning the newly gained knowledge which enables you to build other improvements. Or perhaps accept the found treasure which enables you to speed up the building of an army so you are able to beat the opponent the next round.

Beside these examples this game triggers you to evaluate every situation and create an open and creative mind. When playing this game more often you might be able to select different levels of playing this game. Start on the easiest level when playing first and see if you are able to define you strategy when playing on the hardest level. Like in software testing, initially you might be able to coordinate a simple project. Keep learning from those projects, encrease your skills and see if you are capable to do more difficult projects.

Software testing: An organizational approach (1)

After being involved in several projects over the years I noticed that most of the improvement actions related to software testing are triggered by the testing process it selves. Those improvements are done implicitly or explicitly. Implicitly is done based on the skills of the tester by introducing techniques, strategies to improve the current test process. Explicitly is most of the times done after evaluating the test process and give advice for the next test project.

I think it is not strange that improvements are triggered by the test process, as the influence of the tester is already there. Sometimes, it is only reaching the project it selves.

I noticed that sometimes it is forgotten to look at the impact of those improvements on other processes or the organization. Imagine that you ask as tester you need better documentation. You even claim that you cannot start testing before you have that documentation. This might result in extra work for the developer or business to deliver you request. Only this will result in extending the time of the project as the developer cannot code during writing documentation, or if the developer already started and business comes up with new requirements the project will also delay. In my opinion it is not wrong to ask for better documentation, only keep in mind that the reason for doing this is improving your test process and the organization accepts the impact of it.

In a previous writing I wrote about the focus of testing. See: Do we Test wrong?
The intention of this writing was to pay attention that testing is a process which is a part of the organization. Testing should not be a stand alone goal.

If we look at the test process from an organization view you perhaps might identify the following approached related to software testing, the organization has:
1. one overall approach for software testing;
2. for every project a made to fit approach;
3. for general projects a overall approach and for small projects a made to fit approach;
4. no specific approach for projects, there is continuance of an approach for next releases;
5. every release has its own approach;
6. software testing is not embedded in the organization.

If you also take those views from a development point of view you might find out that the chosen development method might be different also. Assume there are different development methods in the organization like: the basic development method is RUP and some small projects tending to use an Agile approach. You can imagine that this has some impact on the test approach.

And if that test approach is improved only to support that certain test project, the advantages might be within that project, the disadvantages for the organization might be more expensive then those improvements are meant to support.

In most of the organizations a test manager is assigned to monitor all those processes. The question is as always: are choices made correctly?
Perhaps another mean can be introduced: The Organizational Testing Board (I will call it OTB). This should be a staffing function were impact of improvements are determined based on direct and indirect costs and benefits. Considering those costs and benefits the organizational view related to business processes, development methods and test strategies are involved in a improvement plan which is supported by the organization on first hand instead of supporting the individual test process.

If an OTB is established, improvement suggestions are initial comming for the individual test process. They will decide based on costs and benefits for the organization if it is allowed to continue with that improvement. While the decision will be based on the short term and long term plan of (test) process improvement considering the place of chosen development methods in combination with current business processes adapted to the test process and skills of the testers.

Sunday, May 25, 2008

Open System Thinking and Software Testing (7)

This is a continuation of the posting in the category: Open System Thinking and Software Testing. For the previous post you might check out: Open System Thinking and Software Testing (6)

For defining the items and investigation of the relations of those items to each other I'm still working on Micro Level: Test Project. (See Open System Thinking and Software Testing (1) )

As written in the first post, I think you can divide the levels in micro, meso and macro. In the postings 2 until 6 I tried to deal with the test project, micro level.

Let me now try to explain my vision how open system thinking can help with the meso level: Test Process. I will use here also several postings for.

An approach as Open System Thinking can also be used on this meso level. Here you have to pay attention at least on the following questions:
1. Are there improvements suggested by the organization related tot the test process?
2. Are there actions from the test project which results in an improvement activity?
3. Are there other processes which have impact on defining a test strategy?
4. Is there a test process improvement program started?
5. What tooling are available to start a test process improvement program ?
6. What kind of development method is used in the organization?
7. Does the test process fit the existing development program ?
8. Does the development method support the organizational procedures?
9. ...

Of course their are more question which have to be asked to fill in the quadrants of the open system model. Only this will be the start of it.

In the next topics I will perform the same steps as used in micro level to fill in the picture how Open System Thinking can be used helping gaining the overview on test process level.

Again I will perform the following steps:

  1. Define the general meanings of the categories on meso level;
  2. Identify the items per category: Goals, Technology, Culture and Structure;
  3. Fill in the quadrants;
  4. Define how the items are weakening or supporting each other;
  5. Defining the sentences how this empowering/weakening is done;
  6. Defining possible solutions how to monitor or to define new improvement suggestions
If you have some additional remarks or suggestions please leave a comment.

The next article related to this post can be found on: Open System Thinking and Software Testing (8)

Saturday, May 24, 2008

Mikado Management or Test Management

Ever been in a situation were you did not have much control over your test environment? Have you been in such a situation the developers deployed their solutions whenever they thought it was ready? Or have you been in a situation when you came back at work new deployments have been done?

To me it seems that those kinds of situations all previous work you have done on testing was useless when your goal is prove quality instead of finding as much issues in code as there are.

You might compare it with the game: Mikado


In this game every stick has its own color and every color has its own value. When starting you bundle the sticks and let them cautious fall. And your aim is to get the most valuable sticks without disturbing the other sticks. If you translate this to testing every functionality has its own value and you want to see how it works and how it is integrated. Does it disturb other functionality when using it?

Only what happens when new functionality or solutions are offered? All sticks are picked up again and newly setup. The whole stick-pile is transformed and you have to start again. This result into a new situation, sticks which had in earlier phase no direct interaction with other sticks might have now.

Only do you have the time to start testing all things again? Did you handle some kind of regression testing? I think if you keep continuing testing without any additional effort you performing some kind of Mikado Management. You are not aiming for gaining as much points in your time window, you aiming for getting as much sticks in you time window. Your test strategy shifted from proofing quality of main and import/risk full functionality towards finding as much issues in the top shelf of your system.

To deal with situations as this perhaps the following actions can be performed:
1. I would start making sure that you have control of you system under test, you give approval when code can shift in your environment. A disadvantage of this is that some necessary code is not there for you to test
2. Another option is Continuous Integration: it allows you all those transports and delivery time is decreased, only when in this continuous integration also automatic quality measurement is done by the development part, or better: the team, the sticks have not been picked up again, there is prove about the impact of the new code on the system.
3. Or define a set of regression test and after every delivery moment you execute those test set. You can decide based on technical impact of new functionality or solution if you run it immediately or after several deliveries. Or when deliveries are done on daily basis, you can start in the morning with executing that basic set of test cases. Keep in mind that these sets can be dynamically based on the area where solutions are provided and time is under pressure.
4. Another option could be automating your regression set. You can start the regression set when ever a deployment is done.

I think the best situation here could be having the continuous integration mind set. Where it might be an optimal situation where also the regression set of functional test is embedded in this integration and also automatically executed.

Only not every organization or project is yet mature to deal with such an approach. In such a case I suggest to prevent Mikado Management, and keep control or gain control over your test environment. If you have to decide what the quality of the system is, you should be responsible for the state of the system. Therefore you decide when deployments are done. Important in such a case is that management accepts the risk that the test process can extend due to deliver later in time.

And for those dare devils, if you can not get control over the test environment and transports are made when ever. Take the Mikado Game with you and start playing that game instead of testing. Think about what sense does it make if you keep continue testing and you don't get time of performing some regression testing? You loose the picture of quality. In that case playing Mikado you have at least some fun. Playing it with your manager might help make them understand the situation you are into.