Reverse Business Intelligence testing

Currently Business Intelligence is quite hot. Lately there are some shifts in BI-specialized companies as shown in the following article
IBM, Cognos, and the End of Best-of-Breed

Perhaps this will make the live of a tester easier as components will be more integrated. On the other side it might challenge us more in the nearby future as those products might be integrated and placed on the market too swift which might lead in only proven technology when organizations are using the total solutions as they are offered.

What I learned is that organizations always need their own customizations. Sometimes they are forced by habits how the business works; sometimes they are forced by law.

Of course there are several ways of testing these systems. Here an approach which might help to identify if the reporting tool is able to create necessary reports based on existing data. (Keep in mind that this is just a simple description of a possible approach.)

Start testing based on the information needed. And work through the system backwards to identify your starting point.

This might be reached by the following steps:
1. Identify the KPI's
2. Identify the reports these KPI's are involved
3. Prioritize these KPI's
4. Start with the most important KPI
5. Identify the data these KPI consist of
6. Identify the source of those data
7. Identify the quality (appearance) of already existing data related to these KPI's
8. Create a test data set based on the appearance of this data
9. Run the reports

The challenge here is when errors occur, it is hard to say were the error lies, is it in the reporting tool, in the data warehouse or in the source data. At least we know that there are errors which give wrong information about the KPI's. If the location of the error is found and it is related to the (historical) data, then it might be better to correct that data instead of continuing with the next KPI. As those error also might influence the other KPI's.

If there are no unacceptable errors you might continue with the next KPI.

Though this approach will not cover any errors based on data created by the system. For this we might have to extend the test approach by the following steps:
1. Identify the functions of the main system which creates, alters this data
2. Identify the data model and architecture related to these data and functions
3. Define you test cases based on these functions, with this you make sure that data creation is of a certain quality

And continue with testing the next KPI.