Big Data? So what?

I'm delighted to say that my new book, Big Data is now available... but you may think 'So what?' data's not exactly new. What's the fuss all about? This extract from the introduction explains why:

It’s hard to avoid ‘big data’. The words are thrown at us in news reports and from documentaries all the time. But we’ve lived in an information age for decades. What has changed?

Big data has enabled Netflix, for instance, to take risks that conventional networks could never afford - and to target its marketing in a way that was never before possible. And big data is not just about business: among other things, it has the potential to transform policing by predicting likely crime locations; to animate a still photograph; to provide the first ever vehicle for genuine democracy; to predict the next New York Times bestseller; to give us an understanding of the fundamental structure of nature; and to revolutionise medicine.

Less attractively, it means that corporations and governments have the potential to know far more about you, whether to sell to you or to attempt to control you. Don’t doubt it – big data is here to stay, making it essential to understand both the benefits and the risks.

The power of big data derives from collecting vast quantities of information and analysing it in ways that humans could never achieve without computers in an attempt to perform the apparently impossible.

Data has been with us a long time. We are going to reach back 6,000 years to the beginnings of agricultural societies to see the concept of data being introduced. Over time, through accounting and the written word, data became the backbone of civilisation. We will see how data evolved in the seventeenth and eighteenth centuries to be a tool to attempt to open a window on the future. But the attempt was always restricted by the narrow scope of the data available and by the limitations of our ability to analyse it.

Now, for the first time, big data is opening up a new world. Sometimes it’s in a  flashy way with computers like Amazon’s Echo in the home that we interact with using only speech. Sometimes it’s under the surface, as happened with supermarket loyalty cards. What’s clear is that the applications of big data are multiplying rapidly and possess huge potential to impact us for better or worse.

Big Data is available from and