How many bugs do you need to make a metric?

Last year I attended a presentation by Markus Schumacher about metrics. His presentation can be summarized by his saying: "Metrics should change behavior".

I cannot forget about this saying while I'm busy collecting and presenting data. Often I see presentations of figures how many bugs we already have found during testing. Discussions are held how many bugs we didn't find. A lot of time is spent to make nice charts from this data.

Michael Bolton posted a very good blog posting about metrics: Meaningful Metrics

In his blog he also suggests to create metrics which rising questions instead of creating metrics to be used as a control mechanism.

I don't think you can expect from management that you skip presenting control-metrics from the start. It is also not appreciated you don't present any metrics. What I often do is start simple by presenting the number of issues found in a certain period and what their severity is. Although this won't change behavior or raising questions about quality as it should do, it creates the room to discuss what should be measured. Collecting data and creating metrics is also a dynamic process.

I would summarize this process in the following steps:
1. Ask what metrics are needed immediately, let management prioritize because it cost time and effort;
2. Ask what their goals are for those metrics;
3. Define the goal of the selected metrics together with the customer and define criteria how to use them and to monitor the benefits of those;
4. Add some 1 or 2 other metrics you think which are needed to meet their goal;
5. If they got used to those metrics you might even visualize it in charts;
6. After an iteration or a weekly meeting check if it met their goals;
7. Adapt your metrics based on requests, this also can mean you add or remove metrics;
8. Sometimes you have to dive into your collected to see if questions can be defined based on historical data;
9. If people got punished because of the metrics, remove that information as interpreting metrics is a profession on its own and often the metrics we are dealing with can barely be used as indications;
10. Do this every time again, don't rely on the believe if things were good enough the last tie then it should be good enough for now;

Coming back to the title: "How many bugs do you need to create a metric?" I would say: None. You will need at least one question where you have to find an answer to. This answer should lead to change in behavior, make people move or raise more new questions.

Numbers of bugs can be used to create fancy charts. As long as those charts doesn't move people I would say that charts presenting >1000 bugs, 10 categories over a period of 10 weeks are very useful if you want to exercise with colors and chart types.