Testframing: a bit of my perspective

Last week I attended a tutorial on testnet spring event 2011 given by Michael Bolton on Testframing. I already read something about (see: http://www.developsense.com/blog/2010/09/test-framing/ )and now had the chance to see it work. it was a great day as it made me think more about this concept. To preserve my thoughts and ideas I wrote this article. Perhaps you can learn also a bit from it.

Initially I thought it would be an approach were you frame all the tests in constructive sentences, and write them down. So initially it seems to be the same as I was told to do in the past: write test cases and then execute them.

I was wrong then: Why would you write down or use valuable time to write down test cases till the lowest detail when your mind is able to work with the speed of thinking. So there must be more to understand.

During the tutorial I noticed that one of the strength is the way you use a mission to guide the story you want to tell and the observations and actions you do how the story can be told. While explaining there was interaction between the listeners, when something was not clear you change the vocabulary of the proposition or perhaps the vocabulary itself. You even extend the frame using formal or informal connectives

I think that the difference between writing test cases and test framing it takes you much further, it combines the strength of a test strategy with thinking about the justification of testing together with(when necessary real time) adaptation of the path in the system you tending to follow ending up with the explanation of the results you found combined with the justification why you did it. This can be done very fast.

In other approaches like TMap we tend to call it in the Netherlands, there are all kind of delays involved, first the plan, then the written cases, then the execution and after a while a verdict about quality which was stated in the plan.
In some way it is a structured way of working, is it testing? This is another discussion I will not start here. There are other sources about this topic.

In short I learned more about test framing that it is a way of telling a story why and how you tested something combined with the results you found and what you think it tells about the system.
Testframing to me can be done by writing it down, in some cases this makes sense perhaps because of some complexity is involved or some kind of uncertainty about the system you have and you use it to validate your story.
Sometimes you actions are less important and easy to explain afterwards, then you use it without writing it down, just explain it.

Somehow I see a lot of potential in this way of testing. one of the reasons is that it keeps me thinking if I'm doing the right things.

On the same day Michael Bolton also had a keynote were he explained about the dark and the bright future. This was a great view. In the birghter view he explained about first order measurements (some more information can be read at stickyminds.com:
Three Kinds of Measurement and Two Ways to Use Them By Michael Bolton

While explaining first-order-measurement he used a connective "or" this triggered me to think that this is one of the strengths of testframing. You are able to adapt you story based on the first-order-measurements. My first reaction was: This is great!

If I'm able to tell a story about why I'm doing something with what mission (direction) then I'm able to tell the stakeholder/person who cares why Im worth paying for and spending time and resources.
If I'm able to make visible that I'm able to react on circumstances, the information from first-order-measurements, which are happening now instead which happens in the past (the delay when test scripts are written down and the explanation is proved later on) then I provide my self a tool to see if Im on the right way of acting and providing information which are accurate and supports transparency.

Looking back to testframing, I see some different ways to use it with respect to logging the actions:
- use it by thinking in testframing: nothing to write down or explained.
- use it only using your mind, thinking about it and explaining what you have done/doing
- use it in front and write down some notes how you think you want to approach the system
- write down while testing how you framed the mission
- pair testframing: you tell and share your thoughts/mind and another person make notes without interrupting you thinking process (this is another role than the person who challenge you while testing)

Another dimension here is the measurement-order. I think first-order-measurement is an valuable addition to be aware of while framing. Here you can do also some logging as the measurements changed or confirmed the path you were following in your mind.
- before testing and framing first: you might consider which propositions might provide valuable information
- during framing and not logging the testframe itself: log the moment of measurement and result, here you might wonder if you also write down the impact on your story.
- during testframing and also explaining the result: tell the story not only what you have found and why you did it, also explain what impact the measurement had on your frame and ask if you valued your measurements properly. If not there might be some valuable information left behind the framed story.

Somehow I believe testframing is a great way to test a system fast with the right transparency and justification. it is not just a trick you can learn, you have to practice to be able to do it and learn to become better at it. It is a way of thinking and communicating with others which is not only challenge the system as you do it, it also challenge you by explaining how you do it. Perhaps it also tells the story about the tester.

I hope this article challenge you to think a bit on Testframing, for me this is just the beginning in learning about testframing looking for opportunities and time to start with it and learning about it.