It almost feels like people are looking at data with a fresh pair of eyes.
Whether we like the term big data or not, we cannot avoid reading, hearing or talking about it. While using data to better understand what customer think or need, to quantify advertising effectiveness, or to improve supply chain processes, is far from new, a new dimension has been added to data. It almost feels like people are looking at data with a fresh pair of eyes.
Before discussing the new dimension which has been added to data, I’d like to share with you our experience of the last 6 months. An experience that felt like a big data journey. At boobook, we wanted to understand what the real story is behind big data, i.e. is it just noise, as quite a few claim, is it one of these hypes that will eventually die out, or do businesses really believe in it and are they already integrating (big) data into their decision making processes?
We talked to a wide variety of companies, all involved with some part of big data, ranging from market research companies to IT companies, management consultancies and of course the end clients. This lead to very interesting and eye-opening conversations. One thing is clear, big data is definitely talked about a lot. Though despite the noise, a lot of people still do not fully understand it and only a handful of businesses really know what to do with it and are actively using it. But it was very apparent to us that companies do believe that this is much more than a hype. There is no doubt that using data and information will become a real necessity to gain competitive advantage. Companies that really know their customers, not only with regards to their demographic profile, but also in terms of their purchase history and other customer journey interactions, how they communicate, what they communicate about, what they need and believe in, will be able to interact with them in a much more effective way.
There is no doubt that using data and information will become a real necessity to gain competitive advantage.
So why is there still more noise than action?
There are still significant hurdles which have to be overcome. While some companies are struggling with a few, others have a long way to go before they will reach the competitive advantage stage.
There are three main factors that have to be in place before data can be fully translated into value. They are culture, tools and talent. The first one, culture, is most probably the most important one. Companies have to view data as an additional asset. It needs to become part of all strategic decision making and be incorporated in most processes. This means a top down approach and creation of new functional roles. We were very excited to learn that several of the companies we talked to have already appointed a Chief Data Officer (CDO).
While not everyone is ready to start incorporating big data into their decision making process, there is already a wealth of tools on the market that can help with this. In fact, about every week new tools are coming out. The difficulty now is to know which one to use when and for what. Apart from tools, you do of course still need to get your hands on the data. Many businesses believe that when talking about big data, they need to start collecting more data.
This is not necessarily true. Almost all companies we talked to already have a wealth of data. They just do not realise it as it is in different databases scattered around different teams. The silo mentality is still very much alive, though we do notice that this is gradually improving and disappearing in some organisations. So we do advise everyone to start with the data they already have and not collect more for the time being.
While culture is the most important factor to make big data a success, at the moment talent or people is still the most difficult one. Especially when looking for data analysts who are called data scientists these days. And who are data scientists? If we need to believe everything which is written and said about data scientists, these people must have a special gene. They need to be excellent statisticians, very good communicators and visual story tellers, understand the business very well and know about a variety of IT tools. We’d love to find those people! Though to be honest, they do not exist. It does make it clear that any big data project needs a variety of people, i.e. a multi-disciplinary team. This is also not a surprise as most complex business issues or questions are solved by teams of people. Why would this be any different for a big data project?
Nevertheless, the pool of people with in-depth analytical skills, especially senior people, is still small. This will grow over time though.
So what is the extra dimension that has been added to data?
To us, there are three big changes. Some might claim these have been there all the time and they are probably right, but now they have become more widely acknowledged.
Change 1: from one to several data streams which can be tied together to provide a wider view of a customer or process. While before, most of us would either only analyse behavioural data, or sales data or market research data, these days, we have several of these data sources available at the same time. Putting these together, plus even adding mobile (geo data) and social media data does tell you so much more than looking at them separately.
Change 2: from aggregated to individual predictions. We are so used to looking at customers in large segments, e.g. the young versus the old, the big versus the small spenders, etc. With all the data available now, often at individual or very low level, we can communicate with almost every individual in a personalised way, enabling us to use the right language, offer the right product at the right price and at the right time.
Change 3: from quarterly (or even yearly) analysis to weekly, daily or even real time analysis. Looking at data only once or a few times a year does not work anymore. The world around us does move fast, which means that we need to be quick and act at the right time. Given that data is gathered much more frequently, in fact in many occasions continuously, very regular to real time analysis has become possible.
And what have we enjoyed most during our journey?
That people talk about data and want to use it more. At boobook, the big data hype is not about the huge amounts of data being collected, it is about the awareness it has created.
For many years, data sat in a corner gathering dust. Now companies are realising that all this time they have actually been gathering gold-dust.