Though almost a month and half too late, few days back I was browsing the web-pages of individuals and organizations offering predictions around what the IT industry should expect in 2014. Many of them were along expected lines – continued rise of Big Data, Cloud Computing, Mobile Devices, Wearables, Social Media driven Digital Marketing, 3D Printing, Bitcoin etc. Albeit sporadically, there were also predictions around democratization of data analysis i.e. data analysis becoming more of a mainstream task rather than the domain of data scientists. I don’t know about the other predictions, but this one is certainly something I identify with. In the last 5-10 years, “Data driven decisions” has become a buzzword in many organizations. But – look at any organization and it will be apparent that many decisions are not really data driven. There will be a few cases of people doing pivots and some fancy charts using tools like Excel and then there are the data scientists who are supposed to be doing advanced data analysis. Talk to the data scientists and it will be clear that they spend more time on pulling and preparing the data than on any real analysis. And even the analysis mostly centers on some basic statistical techniques like regression and associated visualization done using a tool like R or Matlab.
The primary culprit here is the availability of data and the specialized skills needed to extract the same. This is preventing most people in the organization from validating their decisions with data. The same problem also forces the data scientists from not being able to apply their deep skills – after all the querying and massaging they just have enough time to do some basic analysis. But when we talk about democratization of data analysis, it is not just about accessing data but also enabling somebody well-versed in their job (Sales, Finance, Procurement etc.), to use data optimally without the help of specialists with deeper mathematical background. For most part, the approach to fix this is centered on two type of tools – (1) Self-service BI that is supposed to allow anybody to pull data at will and (2) Automated Pattern Discovery tools that are supposed to help users dig out insights. The former has been around for a while and has had limited success – despite the proliferation of such tools most organizations still struggle to extract the basic data they need. The latter is new and most of the offering in that area came up after “Big Data” became the next big thing!!
At VSSOD we are also into enabling data driven decision making – but for a subset of the organization’s users who deal with day-to-day operations in sales, procurement, logistics and production – essentially the Ops organization, reporting into the Chief Operating Officer. We decided to restrict ourselves because we believe that data analysis has to be tempered with deep functional and industry expertise – out of the box enablement of knowledge “Vedas” as an integral component of data analysis. Given the nature of day-to-day operations we did not have a choice but to make sure that it is available in ‘real-time’. When it comes to pattern discovery we have linked it to key metrics and targets that operations managers are goaled on, so that the signals they see are the ones that affect them and need their immediate attention. Lastly, we recognize the dynamic nature of today’s businesses and have ensured that rules and configurations can be done by business users without relying on data scientists or IT. Will this result in democratization in data for operations – if the story of customers who have adopted OpsVeda so far is anything to go by, it is a BIG YES. Like many others, we also hope that data analysis would be more main-stream in 2014 and operations would be a big success story in that!!!
May 2014 bring you good health, happiness, prosperity and lots of data driven insights.