Testing Schools and Organizational Schools

In the last few days an interesting story was started by Paul Gerrard and James Bach as I mentioned in my previous post: Schools of testing, can you decide? The key in their story is the term Axioms. Can Axioms be defined, used and maintained to help us testers? Do we need Axioms? Can we live without Axioms? Or as Paul Gerrard mentioned in one of the comments: Is Axiom the right word?

Paul Gerrard's replied on both postings with a next posting: Schools Out!

With those articles in mind and their comments on user input I thought about it for this weekend. Somehow, I couldn't get it out of my mind. As was I thinking I'm tending to the Context-Driven school as explained in my previous post. Somehow Paul Gerrard's post makes a lot of sense. It makes sense to let us think about. Or even better, force us to think about.

To understand it all better I tried to paint a picture about it. To understand the object to paint I did a quick search on the internet and found these besides other information these sources about the Schools of testing.

Presentation of Bret Pettichord: Schools of Software Testing these schools are mentioned:

• Analytic School: sees testing as rigorous and technical with many proponents in academia;
• Standard School: sees testing as a way to measure progress with emphasis on cost and repeatable standards;
• Quality School: emphasizes process, policing developers and acting as the gatekeeper;
• Context-Driven School: emphasizes people, seeking bugs that stakeholders care about;
• Agile School: uses testing to prove that development is complete; emphasizes automated testing;

On the Blog of Cem Kaner: Schools of software testing these schools are mentioned:

• Factory school: emphasis on reduction of testing tasks to routines that can be automated or delegated to cheap labor;
• Control school: emphasis on standards and processes that enforce or rely heavily on standards;
• Test-driven school: emphasis on code-focused testing by programmers;
• Analytical school: emphasis on analytical methods for assessing the quality of the software, including improvement of testability by improved precision of specifications and many types of modeling;
• Context-drive school: emphasis on adapting to the circumstances under which the product is developed and used;

They already presented the differences between the schools. Therefore I shall not continue in my own view of schools. What could be interesting is the similarity of these schools. They reflect some timeframe of human thinking and approaching the world.

Something similar also occurred in organizational thinking. In Organization theory, a strategic approach, chapter 2 by Narayanan & Nath they mentioning 3 types of schools:

1. Classical theorists
2. Human relations school
3. Early modernists

Short description of the 3 classical theorist schools: Short description of the 2 Human relations schools:
Short description of the 3 Early modernists schools:

To me it seems that history is repeating in small and much quicker in the Information Technology industry. These schools were defined over the last century while our schools evolve in the last few decades. The challenge might be now how to make our schools fit in the Organizational schools. Here I make the assumption that Organization still tending to one of these schools.

To make testing fit and also work in those difference organizations depends strongly on the skills and experience of the tester. As Testing is getting Global and our occupation has new testers and experience testers it would help that we speak about the same thing. And also be able to explain it to the organization we work for.

In this I see at least 2 levels of communication. One is based how we communicate within our community. This can be done by explaining the principles of our schools. And try to work according to it.

Another level of communicating is making the organization understand what we are doing. This should be translated to the "Organizational School" the organization is based on. To translate we have to explain how testing is helping the organization and what we meant with it. If we are able to this we might be able to get the best out of our schools. I think using Axioms can help to explain the organization what we are doing for them and what we mean with them. Picking the Axiom can be depending on the school we tending to.

Perhaps there is some relation to Law's, we have different kind of laws, they all are captured in the book of laws. Only they apply only to a certain area. Those laws are made to communicate to each other and to make sure we are talking about the same.

To write the book of law's related to software testing would be wrong to do. At least now as we are working in a dynamic environment this needs some kind of flexibility. Though, a book or list of Axioms might help to explain to others what we mean with certain terms and how they can help the organization.

Coming back to my initial questions:

Can Axioms be defined, used and maintained to help us testers? We can define them and use them. Only maintenance would be hard. As who should maintain them.

Do we need Axioms? I think it depends on the skill and experience level of the testers and the understanding of the organization of our occupation.

Can we live without Axioms? I don't think so as there are still projects failing based on commitment of organizations towards obtaining their goal of quality understanding and related to that their undrstanding of the need for us tester's.

Is Axiom the right word? I think it is a better word then Law.

Schools of testing, can you decide?

Today I noticed a posting of James Bach on his blog called The Gerrard School of Testing

In that article he discusses the axioms Paul Gerrard posted on The 12 Axioms of Testing.
James also mentioning the context-driven school and her The Seven Basic Principles of the Context-Driven School

As Dutch is my native language I first had to find out what axiom means. According to Wikipedia, Axiom it means the following: "In traditional logic, an axiom or postulate is a proposition that is not proved or demonstrated but considered to be self-evident. Therefore, its truth is taken for granted, and serves as a starting point for deducing and inferring other (theory dependent) truths."

And this made me think about what school I think I belong to. When starting in the software testing business I learned about one test method. And what that method is saying I took for granted. I assumed it as the initial truth. During practicing I noticed that a method on its own cannot take for granted as it depends on the situation. And sometimes situations seem to look-a-like. Practices from previous projects turned out not completely covering the situation I needed them. They could be best practices, though they need to be altered into new practices.

In all cases the Human Factor was playing a huge role in success of implementing those practices. And beside this Human Factor, also the choice of the organization for development method made a difference. As this lead to the way they intended to communicate in the project as by iterations, increments, documents, delivery moments etc. And what was important to be delivered. In some cases deliverance of a product was a goal in it selves. As some functionality was removed and the initial problem which lead to this development was just partially solved.

This made me think that deliverance of a solution for the problem should be the goal and not deliverance of the product it selves. Though this depends on the situation I was into.

Writing this all down here, I tending to think I could belong to the Context-Driven school, as the principles I believe in fit more in the 7 principles of them. This doesn't make the axioms of Paul Gerrard obsolete. Only would it be short minded to say that those axioms are more the beginning of a vocabulary where instead of explaining the meaning of them the way to get there is explained? That those axioms are communication rules where, what we are talking about?

In the last case for some organizations it would help to talk this language. Though, I'm more tending to say that it depends of the context. Perhaps there are situations where other axioms also work within that organization or project.

Though I'm tending to Context-Driven, as it also supports my thoughts about other articles related to Open System Thinking on my blog, I need to learn more to say that those Axioms are not useful for me.

Cem Kaner made a presentation available related to Context-Driven School which also might worth to take a look at.

[Jeroen: Added on 2-23-2008] Paul Gerrard made another posting Schools Out!. This is worth looking at. It might help us understand how to look at Axioms in respect to Context-Driven School.

I'm not a developer, thought this might help us testers

In the past I ran into an article about visualization of software evolution. It took me some time to dig it up again on the internet.

On this site CVSscan: Visualization of Code Evolution the author ir. Lucian Voinea PDEng gives 3 example tools how to visualize the evolution of code were he claims it can be used as "tool aimed at developers involved in maintenance projects".

Another paper of the same writer on this topic can be found here: Visual assessment of software evolution by L. Voinea, J.J. Lukkien, A. Telea

On his site he explains and offers the following tools:

• CVSscan: Visualization of Code Evolution (file level)
• CVSgrab: Visualization of Code Evolution (project level)
• CSV: Code Structure Visualization
• DreamCode: C++ Abstract Syntax Tree Visualization

You perhaps also want to read the paper he made available from that site called: CVSscan: Visualization of Code Evolution

I think there is also another opportunity here. I think it can be used in Agile/SCRUM projects also. Were writing code is based on usage for that iteration.

Assume that code/functionality changes or better evolves during the project. And as tester you don't immediately know were to look at. Based on turbulence in evolution of code you might decide to test it with more or lesser focus.

Or assume that you first paint a picture with those tools and then measure if the evolution matched the effort you spend on defining Unit tests or other automated tests.

Though I'm quite new on this topic of testing, perhaps others see some advantages of such tools for us testers. Or have already experience with these tools.

Did you explor(e)(atory) testing today?

As often before I search the internet for information. Sometimes (OK, I admit, often) it is related to testing. Usually I come in areas of other weblogs, other articles or presentations. Especially for the last one I use the document format option from Google.

The reason I start search a quest is mostly not predefined. Though their seems to be a certain pattern in it. If I think about it the following situation can be identified:

• A question was asked to me from a colleague for information about software testing;
• A weblog triggered me with a term I want to know more about;
• An article trigger me;
• I have just a question;
• Accidentally;
• Forum;
• New technology;
• ...

Today the search started with a technology I already heard about, and I also tried it in the past. Only this time it was different. In a course I followed in the previous days they showed some great video's which can be found on the internet.

I started now with a certain purpose, only not yet defined, to use Googles-video search. Using one of the obvious phrases is: "Software Testing". The result was that I found an interesting video of a presentation from James Bach "Becoming a Software Testing Expert" . Actually this was the first time I was not scared by the size of it (58 minutes)

During these 58 minutes I heard some terms which made me listen instead watching to the video and start searching further on the internet for more information. (Specifically at those moments James wasn't in the picture, instead a slide) And now and then I looked back to the video to check if the slide was still on. If the slide was still on I continue my search. If James was again in the picture I took a decision. I decided to scroll the video a bit backwards when I thought it was interesting. And sometimes I just continue watching, accepting I missed something.

When the video presentation was finished I raised my self another question: "Would the actual presentation be available on the internet?" Which lead me to the weblog of James Bach were the video and actual presentation can be downloaded.

On that blog I found other triggers which lead me to continue searching, like names James thought it was worth to mention.

I shall not bother you with the other information I came up with. Though this situation made me think. I was exploring the internet for information which should gave me a good feeling I spend my time useful or made me think further for the next step.

And it gave me a good feeling. Not only has it leaded to new information. In some funny kind of way I did some testing today. Currently I would call it Exploratory Testing. As I approached my system, the internet, with a particular goal to find out if could come up with information using my experience and giving me a feeling if I succeeded or not. Were the success was triggered by other information I found during this search.

I had that feeling I was doing some exploratory testing, only was it? Wasn't it some kind of error guessing? In think not. As I was still able to:

• Identify the goal and direction;
• Measure if the goal was reached on every moment;
• Identify the starting point;
• Identify the decision points;
• Reproduce the steps of thinking;
• Create the road map;
• Check if it gave a good feeling;
• Express that feeling;
• Prepare the next step.

Perhaps books, articles, opinions related to exploratory testing would give other results/opinions about this. I think I did my exploratory testing for today. And that felt good.

SCRUM, it hurts because it is interesting?

Over the last 2 days I attended a course for Certified ScrumMaster given by Bas Vodde and Jens Oestergaard.

In the past I read some about Agile and SCRUM. And were involved in an Agile project. I noticed that it is a different game you play as tester. Bas and Jens were able in 2 days to give you a feeling how different this game can be. Come up with examples which can be translated to your situation without the disturbance how you can improve your project into something what looks like SCRUM. They are able to stick to the main thought of SCRUM.

It was also fun to see how they are able to adapt to each other. And how they pulled the best out of the class.

I could give more information about the content of this course. Only then it would be just another source, since there are already enough books about this or weblogs. It is more fun and informative to attend a course like this yourselves.

If you want to attend this course yourself you might check out for dates and locations yourself on Scrum Courses

Some books you might take a look at are:

• Agile Software Development by Alistair Cockburn
• Agile & Iterative Development by Craig Larman
• Agile Software Development by Robert C. Martin
• Extreme Programming Explained by Kent Back
• Agile Software Development with SCRUM by Ken Schwaber and Mike Beedle
• And somewere during this year (2008): Scaling Lean and Agile by Craig Larman and Bas Vodde

Though I learned and heard a lot and in the class and still processing that information. Some slogans of those lessons are:

• Inspect and Adapt
• It depends
• The team decides
• If it gives you a headache it is assumable not your problem
• Facilitating the team
• There is no direct need for a test manager or test coordinator

Especially the last 2 items hit me like a truck. I always thought I was facilitating my teams. I turned that I was politely with best intentions directive towards them. I also thought there is always a need for test managers and test coordinators. Also in SCRUM. They were able to convince me that in an actual SCRUM project there is no need for them. And that hurts.

Though I think we don't become redundant. As there are organizations who embrace SCRUM. There are also organizations were it is not possible (yet). Does this make the course a waist of time? I strongly believe it is NOT. There are practices in the framework which might help me in the future as test coordinator/consultant. And it helps me to share my knowledge about testing within a team. As it depends if you can use it or not. Every organization is unique and every organization needs their customized approaches.

Still I'm looking forward to work with a SCRUM-team.

The Future of Software Testing

Now and then I see and hear questions about what the future of software testing will be.

While starting this post I simultaneously started a search on the internet and find out that this particular question doesn’t stand alone. As it seems I'm not the only one who asks this question and leaves his/her thoughts about it.

Currently (February 10 2008 9:57 AM (GMT+01:00) ) there are about 9,780 hits with this particular search criteria: "The future of software testing"

A presentation by Cem Kraner held on Software Testing Day, Tampere University of Technology, Tampere, Finland, May 4th 2004.

Unfortunately I did not attend his presentation which makes it harder to interpret his ideas. The benefit of this is giving my own thought about it. Assuming future cannot be predicted I think he started correct by not giving answers. Instead he posed questions. Questions will make you think about the item. In my opinion a great statement he made there: "A key risk is the long-term de-skilling of the test group." should trigger us how to look at the future.

Though we cannot oversee the whole community and their impact on our profession. We should be able to create some kind of radar-persons. People who monitor the environment and act when necessary.

Another article I found was written by: Stewart Noakes & Ed Adams wrote a whitepaper in July 3rd 2006 Called: The Future of Software Testing: Over the next five years
In this whitepaper they defined 3 watch words:

• Efficiency
• Value Creation
• Quality build in from the start

Also on seminars this question is getting in the picture as like on EuroSTAR 2008 – “The Future of Software Testing”

On WikiPedia Future is the portion of the time line that has yet to occur, i.e. the place in space-time where lie all events that still will or may occur.

Quote from this site "the stochastic nature of many natural and social processes has made complete forecasting the future impossible"

Assume:

• If complete forecasting of the future is impossible to do;
• Future is controlled by natural processes;
• Future is controlled by social processes.

Combine these assumptions with these quotes:

Eleanor Roosevelt United Nations Diplomat, and First Lady (1933-45), wife of Franklin D. Roosevelt, 32nd US president: "The future belongs to those who believe in the beauty of their dreams."

Malcolm X (1925-1965): "The future belongs to those who prepare for it today."

Perhaps we are not being able to forecast the future completely, though we are able to dream about it and start preparing it today. The preparation could start with:

• Defining our "radar" persons;
• Defining our areas to focus on;
• Become stakeholders of new areas to explore;
• Define how we see our future, not how our future could be based on ideas of "others";
• Define what is best for us as group and also as individual.

Especially the last item could be interesting. Sometimes decisions are taken based on the idea what is good for the organization, like out-sourcing, off-shoring. I wonder if that is good for us. Did we have a vote on it?

We can think about the future. Perhaps we should be careful what we are saying. As it might turn into a self fulfilling prophecy.

Therefore also think what could be good for you and start dreaming.

What is good for you and what are your dreams related to Software Testing?

Open System Thinking and Software Testing (4)

This is a continuation of the posting in the category: Open System Thinking and Software Testing. For the previous post you might check out: Open System Thinking and Software Testing (3)


In the previous post I defined some items to the categories. The next step I want to do is identifying what items form the categories: Goals, Culture, Structure and Technology are supporting or weaken each other.


In the picture below I classified those items.


In the figure below an example is given how to visualize supporting/weakening items.

A table like this gives some lead on those actions were to focus on. A next step could be identifying certain problems which might rise or what positive effect could optimized. For example: Internal testers don't have much experience in testing. This might result in less optimal implementation of a test method. As counter measurement an experienced test coordinator might be assigned to those testers to monitor their work.

Another example would be that not issues are logged as testers are talking to the developers to get some issues solved. Allowing this might result in a less clear overview of the quality as you loose information about issues.

I think a certain result of this excercise not only give information and control over the process. It also gives information about dependancies, boundary conditions and risks which could be used in test plans.

I leave this post as is and will continue with the next post in this category with more examples and perhaps how to continue with it. The postings in this category is a result of a living mind.

The next article related to this post can be found on: Open System Thinking and Software Testing (5)

Change your business process or Software?

Have you been in situations were you have to decide if the software has to be changed or the process? And what do you do?
Most of the times I saw in projects that both did not changed at all. The so called "work around" are defined. Only they are often poorly managed.

You have several types of projects. On one side there is the newly build project. On the other side the fully standard application implementation. And off course all variants in between.
I think there is at least one similarity between these projects. At a certain point you have to decide whether to change the code or the process. Even if you accept the workaround you have to decide after a certain period of time if you change the code or process. In the first situation the workaround is no longer necessary. In the last situation it becomes standard procedure.

On a quest for searching for figures what changes in procedures cost I came to this article:

Which Is Easier: Changing Business Processes or Changing Software? by Loretta M. DeLuca, August 20, 2007

Quote: And you probably don't believe this either, but it really is easier to change your business process rather than your software. Believe me, because I do hate to say, "I told you so."

In this case for us testers there is still a job to do. Though the code doesn't change, you still have to verify that the software indeed matches the business process. The focus will lie on the User Acceptance test.

If the code changes you also have to test the impact of that change onto the system. And the impact of the change on processes. My experience is that certain changes are found in a late stadium of the project. This will make impact analysis mostly focusing on the impact of the system under test rather then measuring the impact on business processes. This lead me to the following article.

Life Cycle for Change Management in Business Processes using Semantic Technologies by Uttam Kumar Tripathi, Knut Hinkelmann and Daniela Feldkamp, Journal of computers, vol. 3, 1 January 2008

Quote: In a fast changing market environment the task Business. Business of reducing the downtime for change management of business processes has high importance. Ensuring that IT reflects the updated business requirement is an important task related to change management in software development.

In that article they defined a process how to manage process changes into Software changes. If I interpreted it correctly they translated the process changes in a formal language and made relationships towards code. I think in according to that paper it is already easier to tell what impact changes in processes have on code. This might help us identify the areas to re-test.

Software Process Improvement from PelicanWeb gives a graphical overview how to measure holes when new requirements are introduced and what the impact is on the project.

Measuring holes and creating new procedures makes me think that we IT-people always tending to re-invent the wheel. Why should be do it when there is more information available related to organizations. In previous lessons I followed related to Organizational Changes I was confronted with a model as described in the article below. This model is written towards organizational changes. Why not use it on detailed level and combine it with the impact towards systems? Perhaps you, reader, have a suggestion how to start?

The Matrix of Change: A Tool for Business Process Reengineering by Erik Brynjolfsson, Amy Austin Renshaw and Marshall van Alstyne, 1997

Quote: The Matrix of Change can help managers identify critical interactions among processes

I know that there are quite good applications to monitor changes in systems, for example cc Harvest or PVCS Dimensions. I'm not sure if they have the capability to monitor changes in processes and procedures.

Perhaps we need another document as well as test basis. A map on how the software is related to requirements and also business processes. It might help to verify if RFC's are judged correctly according their impact. It also might help to identify the impact of changes based on the relationship towards systems. And also when a work around is accepted, what the impact on processes would be. Based on the impact on the process the life cycle of work-around could be defined and also monitored.

Perhaps in Agile projects such a matrix can help identifying the content of the next iteration.

What do you think? Is there already such a framework? Can we use it? Do we need it?

Shuffleboard

Recently a non-IT person asked me what my occupation was. Of course I told him proudly that I'm a software tester. His "silent-Nice" answer indicated that he had no idea about my occupation and why it is challenging.

I asked him if he knew the game Shuffleboard. If he played it once and what he remembered of that game. Now I got his attention. He mentioned that he played it while he was young. A typical game on rainy days with family. Trying to get all the shuffles through the 4 holes on equally basis. He also mentioned that they did it for a period and based on the time available the numbers of rounds are are decided. Sometimes they continued until a certain score was reached.

I explained to him that shuffleboard is similar to software testing. Assume that the shuffles are the number of checks you want to execute with the holes as the target. The area behind the holes is the system. And you are playing it with a team. And based on the available time you can try an undefined number of rounds, or as we can say cycles or iterations.

As you play the game you have your own strategy, perhaps first aim on the hole with the numbers 1 and 2. If they are filled enough you continue to raise you score to equalize the same in the holes 3 and 4. You perhaps use the borders of the board to push the shuffles in the correct hole.

The similarity is that a certain strategy can also be defined in testing. You first define what you want to test, when and how.
What: Our target is to find the holes in the system.
When: when the shuffleboard or system is delivered clean.
How: first the important items, then the less important ones, and still obtaining a good coverage.

If time is over you calculate if you met your goal. If you won is the same as a positive advice to go into production. If you loose calculate the risk to continue or not.

You can say that a game of shuffleboard is similar to software testing as we have our strategies, each situation is different and we are aiming for the holes. And we do it as a team.

Though I couldn't convince him to become a tester, he now understands that I like to play shuffleboard.