Sex, teenagers and Big Data

He who hesitates is lost...
He who hesitates is lost…

A good friend said to me recently that Big Data and analytics is a bit like teenagers and sex; everybody is talking about it but very few are actually doing it. I think he may need to update his knowledge of teenage behaviour but I got his point nonetheless.

The rush to Big Data has the usual hallmarks of other past industry hot trends i.e. lots of hot air and hype. Additionally, there are a lot of definitions of what Big Data actually is and what differentiates it from, say, your bog standard Oracle BI/data warehouse.

So what’s my definition of Big Data? If pushed I’d say something similar to the following: “Big Data is the discipline of analysing vast volumes of structured and/or unstructured data with a view to generating insights and predictions that improve business performance”  (OK, I know that’s not very inclusive of non-business activity but you get the general idea).

My gripe about some soi-disant Big Data companies is that all they have done is moved their dashboard reporting tool to Hadoop (if even that). I can understand the temptation to rebrand an existing BI tool as a Big Data platform but it would be unfortunate if anyone actually fell for that.

Here in Idiro we like to differentiate between BI and predictive analytics – there are many companies offering BI tools of varying levels of sophistication. However, there are far fewer suppliers of predictive analytics platforms  (and even fewer still who provide predictive social network analysis like ourselves). In essence, BI tells you what did happen (i.e. after the horse has bolted) and predictive analytics tell you what will happen (while the horse is still happy in the barn). A smart company will use both.

But back to the definition of Big Data…some would argue that Big Data is all about analysing unstructured data such as blog postings, tweets and other such rubbish. Sorry, yes, I know there is useful information in there but there’s a lot of junk too. We prefer not to discriminate against data and believe that any data can form the input for a Big Data platform.

A word of caution lest anyone think that by installing some Big Data platform that all their problems will be solved. The analytics generated by any such platform need to be used to change business behaviour – this is probably the biggest challenge to the successful deployment of analytics within a company. Often there is political resistance within a company to the use of analytics that makes the Israeli-Palestinian problem seem like a walk in the park. Simply put, people and processes need to change if a company is going to capitalise on its investment in analytics.

As for the aforementioned teenagers, I think that when it comes to the adoption of behaviours that they find “beneficial” they exhibit a lot more openness to change than some large companies who desperately need to reinvent themselves. Big Data may or may not be a panacea for all a company’s problems but, once we step away from the buzz and the hype, what we see is that companies small and large, who intelligently leverage analytics for business really do get the edge over their competitors. Call it Big Data, call it analytics, the important thing is to call it right.

Aidan Connolly
Email a.connolly [at]


Will smartphones continue to be highly viral?

A recent article on smartphones in says:

“Smartphone technology has swiftly reached the point where most future improvements will be incremental. This means that from now on launches of new smartphones will be about as exciting as the latest operating system from Microsoft. This means the primary reason to buy a new smartphone is because you just dropped yours in the toilet.”

Which begs the question – will smartphones continue to be highly viral?  Or, put another way, when people buy smartphones, will this continue to cause numbers of their friends to follow suit, as it currently does?

The iPhone - six years and counting, and still highly viral
The iPhone – six years and counting, and still highly viral

Virality level can be defined as the extent to which one person’s purchase of a product (or any other behaviour change by a person) influences their friends to follow suit.  Some products are highly viral, most are much less so.  Idiro has been measuring the virality of many products, including mobile phones, since before the launch of the first Apple iPhone in 2007.

Over that time, smartphone virality has stayed consistently high, within specific brands and across the smartphone product category.  What this means in practice is that:
– If you buy (for example) an iPhone or a Samsung S4, your friends become much more likely to buy the same model of phone
-Even if they do not, there is a strong likelihood that your friends will be influenced to buy a different brand of smartphone.

Idiro works with mobile operators worldwide to  reduce churn, increase ARPU and acquire customers. In 2008, we won awards for our work on smartphones with Telefonica O2 UK.

So, will the high levels of virality shown by smartphones continue?  It depends on what is driving the virality of smartphones.  Our guess is that in OECD countries it will continue, though declining slowly, for a couple of years at least, until the market for smartphones is saturated and the evolution of apps has stabilised.  The virality of smartphones seems to have as much to do with the extra functionality and benefits as it does with the phones themselves – and that’s down to apps as well as features like NFC.

On the other hand, some new personal gadget could come along with a whole new set of features and benefits.  If such a device can usurp the central role of the smartphone as a personal communication / entertainment / information tool, then the buzz about smartphones will transfer to  them and a whole new wave of virality will begin.

Use of language on Twitter can identify gender?

Social media sites such as Twitter, Tubmlr and Facebook can anonymize the identity of a user, while at the same time enabling them to relate a wide variety of information, comment and opinion. The sometimes spurious link between on-line identity and the message portrayed can be used to impersonate real or fictional personalities, occasionally providing a seeming credible account of current news and events.

However, a recent paper published by John D. Burger and colleagues suggests that a persons use of language can betray certain aspects of their (hidden) identity, most notably their gender.

Their study, entitled “Discriminating Gender on Twitter”, looks at how randomly selected twitter updates (tweets), can identify the gender of a user to an accuracy of approximately 70% [3].

Aside from garnering unwarranted credibility, self-anonymization can lead to a breakdown of social norms, as disentangling a persons “good name” from their identity has been shown to cause them to act with a lessened sense of responsibility [4]. Although the use of language can suggest certain demographic features, it may provide a useful insight into user behaviour when combined with more traditional social network analysis methods.

How many friends are too many?

Social Network Analysis is predicated upon the assumption that people tend to maintain links with their friends over time, by calling, texting, sending IM and otherwise interacting over various forms of Social Media. However, there tends to be an upper limit to the number of meaningful social relationships that can be maintained [1], even through the relative convenience of online networking through Facebook, MySpace, Sina Weibo, Studivz or Twitter to name but a few. A recent article in IEEE Spectrum suggests that on average people are capable of maintaining stable social links within up to approximately 150 people [2]. The benefits of Social Media then, are not in the facilitation of greatly expanded, active social networks (e.g., the average number of friends of users on Facebook is about 120), but in slowing down the “rate of relationship decay by allowing us to keep in touch with friends over long distances”.

[1]- Carl Bialik, “Sorry, You May Have Gone Over Your Limit Of Network Friends”, Wall Street Journal Online, November, 2007

[2]- Robin Dunbar, “How Many “Friends” Can You Really Have?” IEEE Spectrum, June, 2011

Idiro selected as ‘One to watch’ in prestigious Stratecast report

Idiro Technologies has been selected as one of ‘Ten to watch’ in Frost & Sullivan’s Stratecast report on its report “Global OSS/BSS Rat Pack  Stratecast’s 10 To Watch in 2010”.

Commenting on Idiro’s inclusion in the report, Idiro CEO Aidan Connolly stated “Idiro is delighted to be recognised as a leader in this report.  It is a measure of Idiro’s influence on the analytics market”

Within its section on Idiro, the report states:

“This small, but innovative company has garnered multiple accolades in its short tenure. Recent accomplishments of note include: a € 3.56 million grant from Science Foundation Ireland to provide research on consumer interaction and the commercial opportunities that result from the interactions; business with O2 UK and Vodafone in several countries; award-winning projects with Telefonica/O2, in which Idiro demonstrated the ability to reduce churn and understand the viral nature of that CSP’s iPhone sales; and the establishment of sales offices in North America and Latin America, from which the company will seek to expand its revenues.”

The full report can currently be viewed online here.

News: Idiro Technologies Wins Research Grant with Science Foundation Ireland

On the 25th of February 2009, Tánaiste and Minister for Enterprise, Trade and Employment, Mary Coughlan T.D. announced the establishment of 5 new Science Foundation Ireland (SFI) Strategic Research Clusters (SRCs). This represents a €23.9 million investment in ground-breaking, collaborative research activities involving seven academic institutions and 22 companies.

Idiro Technologies is part of the “Clique” Strategic Research Cluster, along with IBM and Norkom Technologies. SFI awarded Clique a grant of €3.56 million. University College Dublin is the lead academic institution for the Clique SRC.

The research offers new insights into how people interact and opens up commercial opportunities. “If a telecommunications company understands their customers in terms of their social behaviour patterns and key influences, they can use this intelligence to cross-sell other products, prevent churning and even build loyalty,” says Aidan Connolly, CEO of Idiro. “We see real commercial benefits to our partnership with Clique. We are interested in commercialising the output of this research so that Idiro can expand to a global scale.”

This research programme is concerned with networks of data about entities and the relationships between them. The most prominent source of such data is social networking sites such as Bebo, Facebook or Myspace but credit-card transactions, mobile phone calls or email exchanges are also readily described as networks.

The availability of this data in electronic format presents some interesting challenges and opportunities for data analysis. Does credit-card fraud have a characteristic pattern of transactions? Can we identify bad behaviour such as spamming or bullying based on the analysis of communications patterns? Can we gain insights into how information disseminates in networks? We can be cautiously optimistic about the answers to these questions but the research challenges are considerable. The volume of data to be analysed is huge and sometimes the patterns are subtle.

Welcoming the announcement, Director-General of SFI, Prof. Frank Gannon said “SFI believes in supporting only top-level relevant research – subsidising mediocre research activity will not produce the outputs required to enhance Ireland’s scientific and innovation landscape and re-energise Ireland’s economy. These new SRCs, have successfully come through a lengthy and thorough peer review. From 40 proposals, I am confident that SFI is supporting the very best teams of researchers and industry-based experts that will help to provide a range of strategic economic benefits to Ireland”.