Big Data and Big Decisions

Big Data and Big Decisions

From a recent article in ASQ’s Quality Progress I came across a fascinating article about ‘Big Data’ by Shu Liu.  To go cliché mad and cut to the chase, Big Data will be a big deal in making big decisions in the future.

Big Data is defined as ‘a collection of data sets that are too big and complex to be processed using traditional database and data processing tools’. It is characterised by the 3 Vs….Volume, Velocity and Variety and is growing exponentially due to ever increasing deployment and capability of technology in every facet of our lives, be it our social media communication, our spending patterns, our medical histories, our diets,  our travel history, etc.

Volume refers to the fact that in 2013 there was 1.2 zettabytes of data (a zettabyte is 10 to the power of 23) and is estimated to exceed 10 times that amount early this year.

Velocity refers to the increasing speed at which data is generated, collected, processed and stored

Variety is in the sources of data that exist and is also growing….databases, documents, emails, phone and text records, social media records, audio and video flies, transactional records, etc.

The author makes a very valid point in that if big data is not managed properly it may pose a threat to our privacy, security and indeed democracy.

So what has all this got to do with Continuous Improvement and Industry? Well in a nutshell, Data (big or small) is the lifeline of a CI Program and likewise if not managed strategically and prudently ‘Big Data’ may also hinder our ability to identify opportunities for improvement and cost savings, manage our processes effectively and efficiently to meet every increasing customer and market demands. How many times have you heard your colleagues say? ….Sure we have no shortage of data, we just don’t know how to extract it, analyse and make sense of it….that is the 1st sign of the Big Data Phenomena and it is not likely to diminish anytime soon!

Submitted by our Lean Six Sigma tutor, Éamon Ó Béarra

Full details of our Lean Six Sigma training courses are available here