Feeds:
Posts
Comments

Archive for the ‘Visualization’ Category

Ken Rona tweeted earlier today about a subject which stuck a chord. He was writing about the difference between a business analyst and data analyst (or data scientist as they are increasingly called). I wanted to expand on the idea as It is important to distinguish between the two roles. I have seen a lot of confusion around the definitions and some executives thinking they are one and the same and others who believe they are totally different. The truth is probably somewhere in between. Here is my attempt at comparing along the key skill set dimensions:

Business Analyst Data Scientist
Business domain knowledge Expertise in industry domain Very good working knowledge of industry domain
Data handling
skills
Ability to handle multiple CSV files and import them into Access or Excel for analysis Ability to write SQL queries to extract data from databases and join multiple datasets together
Analytics skills Knowledge of simple business statistics(statistical significance, sampling), Able to use statistics functions in Excel Proficiency in advanced mathematics/statistics (regressions, optimization, clustering analysis etc.)
Insight presentation skills Storytelling skills using PowerPoint Storytelling skills using information visualization and PowerPoint
Problem solving
skills
Proficiency in hypotheses driven approach is good to have Proficiency in hypotheses driven approach is must have
Tools Access, Excel, PowerPoint etc MS SQL, Oracle, Hadoop, SQL SAS, SPSS, Excel, R, Tableau etc

I think this is a good starting point but can be refined further. Feedback/comments are welcome.

Read Full Post »

The central thesis of my work for last few years has been focusing on the rise of integrated decision support systems which combine data, analytics and visualization to solve a very specific problem extremely well. These systems are usually prescriptive (instead of predictive) in nature and prove to be game changers in their space.

I recently came across one company that neatly falls into this category. They are called ValueAppeal and they answer a very specific question:

Are you paying too much in property taxes?

ValueAppeal saves property owners thousands by evaluating their property taxes and then guiding them through a simple 3-step process to create a custom appeal. The key to our process is our proprietary Assessment Analyzer.

Enter your address in our free Assessment Analyzer. Using the same official data the assessor uses, ValueAppeal’s proprietary algorithms dig deep to determine if your home is over-assessed and how much you can save on property taxes!

For a one-time fee of $99 (with money back guarantee) the company has seen average savings of close to $900

There are three things which are very interesting  and clever about the solution:
1) it uses the available public data and not any proprietary dataset
2) the analytical algorithm focuses on the datapoints that helps to make a favorable case
3) it generates a report that can be simply printed out a dropped into the mail by the user

Please drop a note if you have seen other interesting companies which develop such niche analytics based decision support solutions.

Read Full Post »

Interesting stories on information & decisions that influenced my thinking and were tweeted by me:

  1. Extending the value of operational #data to serve your customers. Netflix ISP comparison http://tinyurl.com/4whjd3o
  2. Coal economics and computer chips. Demand driver for need for more #analytics http://bit.ly/f6Hme2
  3. Excellent paper for #analytics practitioners on customer lifetime value and RFM models http://bit.ly/dkdkaa
  4. #visualization of loobying efforts http://reporting.sunlightfoundation.com/lobbying/
  5. #Analytics of dating http://blog.okcupid.com/
  6. LinkedIn’s @PeteSkomoroch on the key skills that data scientists need. http://oreil.ly/hXZTVJ

Read Full Post »

Theme 5: Good data visualization leads to smarter decisions.

The above visualization has arguably saved more lives than all predictive analytics initiatives combined.  It is visualization created in 1854 by Dr. John Snow. As Miles Dowsett has written,

In 1854 London was the biggest city the world had ever seen, and was literally drowning in its own filth. There was no sewage system in place and those that lived in the capital literally threw their waste and cesspits out into the crowded streets and river Thames. As a result, London was a disgustingly smelly place to be and was periodically engulfed by disease – most notably Cholera.

However at that time it was believed that it was the smell of London that was the root cause of diseases such as cholera. It came to be known as the Miasma theory of disease. However, Dr. Snow was skeptical of the Miasma theory of smell causing disease and in his 1849 essay On the Mode of Communication of Cholera actually introduced the water borne theory of Cholera. He presented his findings to London’s health authorities but couldn’t convince anybody of its merit, and was largely ignored.

In August, 1854 there was another outbreak of cholera in the Broad Street neighborhood. Over a 10 day period cholera decimated the population; 10% of the neighborhood died.

Since Dr. Snow was convinced that Cholera was a waterborne virus, it didn’t take him very long to identify that it was the infected water pump at 40 Broad Street that caused the epidemic.  Instead of solely relying on numbers, he produced a visual of the Broad street neighborhood where he marked the water pump at 40 Broad Street and designated each cholera death with a bar. The concentration of deaths around the pump at 40 Broad Street, which trailed off the further out from the pump one went, was so convincing that the authorities finally accepted the theory of Cholera being water borne. They removed the handle of the pump and it ultimately stopped the spread of the disease in 1854. It also paved the way for a sewage and sanitation systems to be put in place; one of the greatest engineering feats to be undertaken in London’s history, changing the way that urban systems exist and continue to grow to this day.

As the story demonstrates, good data visualization leads to smarter decisions.  Far too often, analysts focus all their efforts on data collection and modeling and pay very little attention to presenting the results in a way that decision makers can relate to.  Every successful predictive analytics project leads to a change from the status quo way of doing things, and it is never easy to convince the decision makers that the new way is better.  Most decision makers need more than R square or a mean absolute percentage error metric to be convinced about the efficacy of a solution based on predictive analysis and feel comfortable approving the change. This is where data visualization skills become important.

James Taylor wrote that visualization is more relevant in context of strategic decisions and not so much for the operational decisions.

Decision making at the operational level is too high-speed, too automated for much in the way of visualization to be useful at the moment of decision.

While I do agree with him, I know of instances where visualization has been creatively used in the very operational environment of call centers. Speech analytics and emotion detection are growing areas in call center technology where depending upon the choice of words, the speech analytics system detects the emotion level of the caller and displays an appropriate emoticon on the agent’s desktop right when the call is transferred to them. Even without understanding the complexity of the caller’s issue, the agent immediately gets a guidance about the emotional state for the caller.

As consultants we are always trying to create that ‘money-visual’ in our presentations, a slide which brings all the analyses together and unambiguously calls out for a need to change or drive action. I feel every predictive analytics project needs one such ‘money-visual’.

Do you have any examples of visuals which you have used to convince people to drive change?

Here are the links to previous postings: part 1, part 2, part 3 and part 4

Cross-posted on TheInfromationAdvantage.com

Read Full Post »

Earlier this week, I suggested a potential business application of IRS’s internal migration data for a moving and relocation company.

Folks at Neilsen Claritas just found a far more interesting correlation which should have driven a lot of business decisions.  They note:

Today’s presence of underwater mortgages, or homes with negative equity, seem to be correlated to two common regional U.S. population trends: 1) domestic immigration from the Northeastern region to the South and Southwestern regions of the U.S., and 2) migration from coastal California inland

While such retrospective analysis is interesting for reports and blogs, it is not particularly useful for businesses. Maybe as means to generate  interesting hypothesis for future. It would have been useful had the chart been available to the strategic planning or risk group of businesses signing up people for these housing loans in 2006 and 2007.

Data is valuable only when it is used to drive decisions. Most companies have a huge opportunity to do a better job in bringing together data, analytics and visualization and delivering them to the points of decision.

Read Full Post »

I believe that the next wave of productivity improvement in companies is going to come from investment in decision support tools.

In our economy almost all workers are now knowledge workers. However unlike the workers of the industrial era, we still do not have the right set of tools to support our knowledge workers. We live in an era of information overload where employees increasingly need to make faster and more complex decisions using large amounts of available data. Under such circumstances, making informed, let alone optimal decisions is simply not humanly possible.

This creates a need for a range of new tools for the employees. Tools that will guide the decision making process and where ever appropriate automate them. To create such tools, companies will need to create expertise in four foundational areas:

1. Data: Identifying, collecting, managing and curating the data within the company and relevant third party sources
2. Analytics: Creating a scalable process to turn data to relevant insights and recommendations
3. Visualization: Presenting the insights within the appropriate context to support a decision
4. Integration: Bringing all of the pieces together to make the recommendation/insight available at the point of decision in the workflow of the employee

Companies will become better at bringing together the four foundational areas. We will also see increased activity in the vendor space. The established ones becoming more active in acquiring companies in the value chain. And a range of startups who will rush into the space to fill the gap.

For now, the blog is about tracking this trend.

Read Full Post »

Follow

Get every new post delivered to your Inbox.