Empowering Modular Automation to make data-driven decisions

A good friend said to me recently that Big Data and analytics is a bit like teenagers and sex; everybody is talking about it but very few are actually doing it. I think he may need to update his knowledge of teenage behaviour but I got his point nonetheless.

The rush to Big Data has the usual hallmarks of other past industry hot trends i.e. lots of hot air and hype. Additionally, there are a lot of definitions of what Big Data actually is and what differentiates it from, say, your bog standard Oracle BI/data warehouse.

So what’s my definition of Big Data? If pushed I’d say something similar to the following: “Big Data is the discipline of analysing vast volumes of structured and/or unstructured data with a view to generating insights and predictions that improve business performance”  (OK, I know that’s not very inclusive of non-business activity but you get the general idea).

My gripe about some soi-disant Big Data companies is that all they have done is moved their dashboard reporting tool to Hadoop (if even that). I can understand the temptation to rebrand an existing BI tool as a Big Data platform but it would be unfortunate if anyone actually fell for that.

Here in Idiro we like to differentiate between BI and predictive analytics – there are many companies offering BI tools of varying levels of sophistication. However, there are far fewer suppliers of predictive analytics platforms  (and even fewer still who provide predictive social network analysis like ourselves). In essence, BI tells you what did happen (i.e. after the horse has bolted) and predictive analytics tell you what will happen (while the horse is still happy in the barn). A smart company will use both.

But back to the definition of Big Data…some would argue that Big Data is all about analysing unstructured data such as blog postings, tweets and other such rubbish. Sorry, yes, I know there is useful information in there but there’s a lot of junk too. We prefer not to discriminate against data and believe that any data can form the input for a Big Data platform.

A word of caution lest anyone think that by installing some Big Data platform that all their problems will be solved. The analytics generated by any such platform need to be used to change business behaviour – this is probably the biggest challenge to the successful deployment of analytics within a company. Often there is political resistance within a company to the use of analytics that makes the Israeli-Palestinian problem seem like a walk in the park. Simply put, people and processes need to change if a company is going to capitalise on its investment in analytics.

As for the aforementioned teenagers, I think that when it comes to the adoption of behaviours that they find “beneficial” they exhibit a lot more openness to change than some large companies who desperately need to reinvent themselves. Big Data may or may not be a panacea for all a company’s problems but, once we step away from the buzz and the hype, what we see is that companies small and large, who intelligently leverage analytics for business really do get the edge over their competitors. Call it Big Data, call it analytics, the important thing is to call it right.

Aidan Connolly
Email: a.m.connolly@idiro.com