Theme 4: Statistical techniques and tools are not likely to provide competitive advantage
I read this interesting post from Sijin
describing his journey to master a video game (emphasis added by me)
All this kind of reminded me of my experiences with finding the perfect weapon while playing Call of Duty 4 over the past year. I spent 3 hours a day almost every day for the past one year playing this game, reaching the max prestige level (the “elite” club) in the multi-player version. I became really good at it… no matter what weapon I was using. But I remember when I started out and I really sucked, I became obsessed with finding the perfect weapon with the perfect set of perks and add-ons. I used to wander the forums asking people about which weapons and perks to use on which map and what the best tips were etc. Thinking that having the perfect weapon would make me a good player. In the end, the only thing that mattered was all the hours I put in to learn all the maps, routes, tricks and my ability (I like to think). The surprising thing was that once I mastered the game, it didn’t really matter what weapon I chose, I was able to adapt any weapon and do a decent job.
This story captures the essence of the theme of this post.
The popular statistical techniques frequently used in business analytics like linear regression and logistic regression are more than half-a-century old. System dynamics was developed in 1950s. Even neural networks have been around for more than 40 years. SAS was founded in 1976 and the open source statistical tool R was developed in 1993. The point is that popular analytical techniques and tools have been around for some time and their benefits and limitations are fairly well understood.
An unambiguous definition of the business problem that will impact a decision, a clear analysis path leading to output, thorough understanding of various internal and 3rd-party datasets are all more important aspects of a predictive analytics solution than the choice of the tool. Not to mention having a clear linkage between the problem, the resulting decision, and measurable business value. The challenge is in finding an expert user who understands the pros and cons and adapts the tools and techniques to solve the problem at hand. Companies will be better served by investing in the right analytical expertise rather than worrying about the tools and technique as the right analytical team can certainly be a source of competitive advantage.
While this theme is fairly well understood within the analytics practitioner community, the same cannot be said about business users and executives. It is still easy to find senior executives who believe that ‘cutting edge’ techniques like neural networks should be used to solve their business problem or predictive analytics tools are a key differentiator while selecting analytics vendors. The analytics community needs to do a better job in educating the business user and senior executives about this theme.
You can read the previous installments of the series here (part 1, part 2, and part 3).
Cross-posted on TheInformationAdvantage.com
Read Full Post »
Theme 2: Modeling Strategic vs. Operational Decisions
In the first post of this eight part series, I wrote about the importance of understanding the cost of a wrong decision prior to making an investment in a predictive modeling project.
Once we determine the need for the investment, we need to focus on type of modeling approach. The modeling approach depends on the type of decision that we want predictive model to drive. Decisions can be broadly categorized as either operational or strategic .
I define operational decisions as those that have a specific and unambiguous ‘correct answer’, whereas in strategic decisions an unambiguous ‘correct answer’ is not available. Moreover such decisions have a cascading effect on adjacent and related decisions of the system.
Think about a health plan predicting a fraudulent insurance claim versus predicting the impact of lowering reimbursement rates for patient re-admissions to hospitals.
An insurance claim is either fraudulent or not. The problem is specific and there is an unambiguous correct answer for each claim. Most transaction level decisions fall in this category.
Now consider the second problem. Lowering the reimbursement rate of patient readmission will certainly incent the physician and hospitals to focus on good patient education, follow-up outpatient care and to ensure complete episodes of care for the patient during their time in the hospital. This should result in lower cost for the health plan. However, it can also lead to hospitals delaying the discharge of patients during their first admission or physicians treating patients in an out-patient setting when they should be at the hospital and ending up in emergency room visits. This is the cascading effect of our first decision that will increase the cost of care. Strategic decisions have multiple causal and feedback loops which are not apparent and an unambiguous right answer is hard to figure. Most policy decisions fall in this category.
The former is an operational decision and requires established statistical (regression, decision tree analysis etc.) and artificial intelligence techniques (e.g. neural networks, genetic algorithms). The key focus of the problem is to predict whether a claim is fraudulent or not based on historical data. Understanding the intricacies of causal linkages is desirable but not necessary (e.g. neural networks). The latter needs predictive modeling approaches that are more explanatory in nature. It is critical to understand causal relationships and feedback loops of the system as a whole. The idea is to develop a model which accurately captures the nature and extent of relationships between various entities in the system based on historical data, to facilitate testing of multiple scenarios. In the re-admission policy example, such a model will help determine the cost impact based on the various scenarios of provider adoption and behavior change (percentage of providers and hospitals that will improve care vs. those that will not adapt to the new policy). Simulation techniques like systems dynamics, agent based models, monte carlo and scenario modeling approaches are more appropriate for such problems.
Bottom line, it is important to remember that strategic and operational decisions need different predictive modeling approaches and the two questions you have to ask yourself:
- Is the decision you want to drive operational or strategic in nature?
- Are you using the appropriate modeling approach and tools?
Cross-posted on TheInformationAdvantage blog
Read Full Post »