Here is my hypothesis of why it is so:
There is a fundamental Catch 22 situation in most large companies. Organizations do not have consistently good quality data (mainly due to process issues during intake) and unless the data is used to making real business decisions, it is hard to improve its quality.
This Catch 22 can only be resolved by very senior executive (read C level) who commits himself to making decisions and measuring performance based on analysis done with imperfect data (but good enough for many types of decisions/relative measurements). Once middle management understands how the data is being used, it spurs process changes to fix the quality issue which in turns increases the accuracy and reliability of analysis. The virtuous cycle is key for large companies which ‘compete on analytics’
In contrast, the middle management never wants to be in a situation to justify their decisions knowingly made using imperfect data. It is easier to justify subjective gut feel than objective decisions made with data with known quality issues.
In summary – the culture of analytics is a top down phenomenon
What do you think? Do you agree with this observation?