How to find gold in Big Data

A CFO I worked for once famously said; ‘give me the golden nugget, don’t give me all the dirt’. What he meant is that although he knew the finance communicate navigates its way every month through a lot of data, he was only interested in key insights. Presented in an understandable way, without too many numbers please.

golden nugget

Makes sense doesn’t it? Well not when we talk about big data. It often seems that big data is the main topic of interest, whilst it should be all about what we do with that data. When I started my career 15 years ago at IBM, data mining was a hot topic. Since then a lot has changed; the whole world got connected, computers got faster, algorithms got better and we spit out uncontrollable mountains of data every second. The process of what we do with data hasn’t really changed though.

The process goes something like this; we have a mountain of data, it might be even big data. We scroll through it and try and find connection with smart algorithms to create information. With this information we might test a hypothesis or create a business case. We then assess the value of this. We then present our perceived value to a group of decision makers to recommend something. This group of decision makers, often with different opinions and agendas have to believe our perceived value and decide to progress. We hope this is the right decision. Further progress can be further research or executing the recommendation. Only when we have executed we can assess if the value of our data analysis actually paid a return on our efforts.

Get the picture? A lot has to go right before big data makes a return. And of course it can and will when done right. We just shouldn’t lose sight of the importance of ‘the golden nugget’. In an IBP cycle you can practice every month to distil a little mountain of information and present the key insights and the right recommendations. This is hard enough and making the right decisions is hard enough too.

Maybe we should focus a little bit more on presenting golden nuggets and making the right decisions whilst we’re figuring out what to do with big data.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s