Big Data in Practice E-Book

Bernard Marr

• Herausgeber: John Wiley & Sons
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 354

Veröffentlichungsjahr: 2016

Beschreibung

The best-selling author of Big Data is back, this time with a unique and in-depth insight into how specific companies use big data. Big data is on the tip of everyone's tongue. Everyone understands its power and importance, but many fail to grasp the actionable steps and resources required to utilise it effectively. This book fills the knowledge gap by showing how major companies are using big data every day, from an up-close, on-the-ground perspective. From technology, media and retail, to sport teams, government agencies and financial institutions, learn the actual strategies and processes being used to learn about customers, improve manufacturing, spur innovation, improve safety and so much more. Organised for easy dip-in navigation, each chapter follows the same structure to give you the information you need quickly. For each company profiled, learn what data was used, what problem it solved and the processes put it place to make it practical, as well as the technical details, challenges and lessons learned from each unique scenario. * Learn how predictive analytics helps Amazon, Target, John Deere and Apple understand their customers * Discover how big data is behind the success of Walmart, LinkedIn, Microsoft and more * Learn how big data is changing medicine, law enforcement, hospitality, fashion, science and banking * Develop your own big data strategy by accessing additional reading materials at the end of each chapter

Leseprobe

“Amazing. That was my first word, when I started reading this book. Fascinating was the next. Amazing, because once again, Bernard masterfully takes a complex subject, and translates it into something anyone can understand. Fascinating because the detailed real-life customer examples immediately inspired me to think about my own customers and partners, and how they could emulate the success of these companies. Bernard's book is a must have for all Big Data practitioners and Big Data hopefuls!”

Shawn Ahmed, Senior Director, Business Analytics and IoT at Splunk

“Finally a book that stops talking theory and starts talking facts. Providing real-life and tangible insights for practices, processes, technology and teams that support Big Data, across a portfolio of organizations and industries. We often think Big Data is big business and big cost, however some of the most interesting examples show how small businesses can use smart data to make a real difference. The businesses in the book illustrate how Big Data is fundamentally about the customer, and generating a data-driven customer strategy that influences both staff and customers at every touch point of the customer journey.”

Adrian Clowes, Head of Data and Analytics at Center Parcs UK

“Big Data in Practice by Bernard Marr is the most complete book on the Big Data and analytics ecosystem. The many real-life examples make it equally relevant for the novice as well as experienced data scientists.”

Fouad Bendris, Business Technologist, Big Data Lead at Hewlett Packard Enterprise

“Bernard Marr is one of the leading authors in the domain of Big Data. Throughout Big Data in Practice Marr generously shares some of his keen insights into the practical value delivered to a huge range of different businesses from their Big Data initiatives. This fascinating book provides excellent clues as to the secret sauce required in order to successfully deliver competitive advantage through Big Data analytics. The logical structure of the book means that it is as easy to consume in one sitting as it is to pick up from time to time. This is a must-read for any Big Data sceptics or business leaders looking for inspiration.”

Will Cashman, Head of Customer Analytics at AIB

“The business of business is now data! Bernard Marr's book delivers concrete, valuable, and diverse insights on Big Data use cases, success stories, and lessons learned from numerous business domains. After diving into this book, you will have all the knowledge you need to crush the Big Data hype machine, to soar to new heights of data analytics ROI, and to gain competitive advantage from the data within your organization.”

Kirk Borne, Principal Data Scientist at Booz Allen Hamilton, USA

“Big Data is disrupting every aspect of business. You're holding a book that provides powerful examples of how companies strive to defy outmoded business models and design new ones with Big Data in mind.”

Henrik von Scheel, Google Advisory Board Member

“Bernard Marr provides a comprehensive overview of how far Big Data has come in past years. With inspiring examples he clearly shows how large, and small, organizations can benefit from Big Data. This book is a must-read for any organization that wants to be a data-driven business.”

Mark van Rijmenam, Author Think Bigger and Founder of Datafloq

“This is one of those unique business books that is as useful as it is interesting. Bernard has provided us with a unique, inside look at how leading organizations are leveraging new technology to deliver real value out of data and completely transforming the way we think, work, and live.”

Stuart Frankel, CEO at Narrative Science Inc.

“Big Data can be a confusing subject for even sophisticated data analysts.  Bernard has done a fantastic job of illustrating the true business benefits of Big Data.  In this book you find out succinctly how leading companies are getting real value from Big Data – highly recommended read!'

Arthur Lee, Vice President of Qlik Analytics at Qlik

“If you are searching for the missing link between Big Data technology and achieving business value – look no further! From the world of science to entertainment, Bernard Marr delivers it – and, importantly, shares with us the recipes for success.”

Achim Granzen, Chief Technologist Analytics at Hewlett Packard Enterprise

“A comprehensive compendium of why, how, and to what effects Big Data analytics are used in today's world.”

James Kobielus, Big Data Evangelist at IBM

“A treasure chest of Big Data use cases.”

Stefan Groschupf, CEO at Datameer, Inc.

BIG DATA IN PRACTICE

HOW 45 SUCCESSFUL COMPANIES USED BIG DATA ANALYTICS TO DELIVER EXTRAORDINARY RESULTS

BERNARD MARR

This edition first published 2016

© 2016 Bernard Marr

Registered officeJohn Wiley and Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book and on its cover are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher and the book are not associated with any product or vendor mentioned in this book. None of the companies referenced within the book have endorsed the book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data is available

A catalogue record for this book is available from the British Library.

ISBN 978-1-119-23138-7 (hbk)  ISBN 978-1-119-23139-4 (ebk) ISBN 978-1-119-23141-7 (ebk)  ISBN 978-1-119-27882-5 (ebk)

Cover Design: Wiley Cover Image: © vs148/Shutterstock

This book is dedicated to the people who mean most to me: My wife Claire and our three children Sophia, James and Oliver.

CONTENTS

INTRODUCTION

What Is Big Data?

Big Data Opportunities

1: WALMART: How Big Data Is Used To Drive Supermarket Performance

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

2: CERN: Unravelling The Secrets Of The Universe With Big Data

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

3: NETFLIX: How Netflix Used Big Data To Give Us The Programmes We Want

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

4: ROLLS-ROYCE: How Big Data Is Used To Drive Success In Manufacturing

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

5: SHELL: How Big Oil Uses Big Data

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

6: APIXIO: How Big Data Is Transforming Healthcare

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

7: LOTUS F1 TEAM: How Big Data Is Essential To The Success Of Motorsport Teams

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

8: PENDLETON & SON BUTCHERS: Big Data For Small Business

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

Notes

REFERENCES AND FURTHER READING

9: US OLYMPIC WOMEN’S CYCLING TEAM: How Big Data Analytics Is Used To Optimize Athletes’ Performance

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

10: ZSL: Big Data In The Zoo And To Protect Animals

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

11: FACEBOOK: How Facebook Use Big Data To Understand Customers

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

12: JOHN DEERE: How Big Data Can Be Applied On Farms

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

13: ROYAL BANK OF SCOTLAND: Using Big Data To Make Customer Service More Personal

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

14: LINKEDIN: How Big Data Is Used To Fuel Social Media Success

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

15: MICROSOFT: Bringing Big Data To The Masses

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

16: ACXIOM: Fuelling Marketing With Big Data

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

17: US IMMIGRATION AND CUSTOMS: How Big Data Is Used To Keep Passengers Safe And Prevent Terrorism

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

18: NEST: Bringing The Internet of Things Into The Home

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

19: GE: How Big Data Is Fuelling The Industrial Internet

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

20: ETSY: How Big Data Is Used In A Crafty Way

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

21: NARRATIVE SCIENCE: How Big Data Is Used To Tell Stories

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

22: BBC: How Big Data Is Used In The Media

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

23: MILTON KEYNES: How Big Data Is Used To Create Smarter Cities

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

24: PALANTIR: How Big Data Is Used To Help The CIA And To Detect Bombs In Afghanistan

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

25: AIRBNB: How Big Data Is Used To Disrupt The Hospitality Industry

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

26: SPRINT: Profiling Audiences Using Mobile Network Data

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

27: DICKEY’S BARBECUE PIT: How Big Data Is Used To Gain Performance Insights Into One Of America’s Most Successful Restaurant Chains

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

28: CAESARS: Big Data At The Casino

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

29: FITBIT: Big Data In The Personal Fitness Arena

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

30: RALPH LAUREN: Big Data In The Fashion Industry

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

31: ZYNGA: Big Data In The Gaming Industry

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

32: AUTODESK: How Big Data Is Transforming The Software Industry

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

33: WALT DISNEY PARKS AND RESORTS: How Big Data Is Transforming Our Family Holidays

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

34: EXPERIAN: Using Big Data To Make Lending Decisions And To Crack Down On Identity Fraud

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

35: TRANSPORT FOR LONDON: How Big Data Is Used To Improve And Manage Public Transport In London

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

36: THE US GOVERNMENT: Using Big Data To Run A Country

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

37: IBM WATSON: Teaching Computers To Understand And Learn

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

38: GOOGLE: How Big Data Is At The Heart Of Google’s Business Model

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

39: TERRA SEISMIC: Using Big Data To Predict Earthquakes

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

40: APPLE: How Big Data Is At The Centre Of Their Business

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

41: TWITTER: How Twitter And IBM Deliver Customer Insights From Big Data

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

42: UBER: How Big Data Is At The Centre Of Uber’s Transportation Business

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

43: ELECTRONIC ARTS: Big Data In Video Gaming

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

44: KAGGLE: Crowdsourcing Your Data Scientist

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

45: AMAZON: How Predictive Analytics Are Used To Get A 360-Degree View Of Consumers

Background

What Problem Is Big Data Helping To Solve?

How Is Big Data Used In Practice?

What Were The Results?

What Data Was Used?

What Are The Technical Details?

Any Challenges That Had To Be Overcome?

What Are The Key Learning Points And Takeaways?

REFERENCES AND FURTHER READING

FINAL THOUGHTS

ABOUT THE AUTHOR

ACKNOWLEDGEMENTS

Index

EULA

Guide

Cover

Table of Contents

1

Pages

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

17

18

19

20

21

22

23

25

26

27

28

29

30

31

32

33

34

35

37

38

39

40

41

42

43

45

46

47

48

49

51

52

53

54

55

57

58

59

60

61

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

81

82

83

84

85

87

88

89

90

91

92

93

95

96

97

98

99

100

101

103

104

105

106

107

108

109

111

112

113

114

115

117

118

119

120

121

122

123

125

126

127

128

129

131

132

133

134

135

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

157

158

159

160

161

163

164

165

166

167

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

189

190

191

192

193

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

217

218

219

220

221

223

224

225

226

227

228

229

230

231

232

233

234

235

237

238

239

240

241

243

244

245

246

247

248

249

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

267

268

269

270

271

273

274

275

276

277

278

279

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

297

298

299

301

302

303

304

305

306

307

308

INTRODUCTION

We are witnessing a movement that will completely transform any part of business and society. The word we have given to this movement is Big Data and it will change everything, from the way banks and shops operate to the way we treat cancer and protect our world from terrorism. No matter what job you are in and no matter what industry you work in, Big Data will transform it.

Some people believe that Big Data is just a big fad that will go away if they ignore it for long enough. It won’t! The hype around Big Data and the name may disappear (which wouldn’t be a great loss), but the phenomenon will stay and only gather momentum. What we call Big Data today will simply become the new normal in a few years’ time, when all businesses and government organizations use large volumes of data to improve what they do and how they do it.

I work every day with companies and government organizations on Big Data projects and thought it would be a good idea to share how Big Data is used today, across lots of different industries, among big and small companies, to deliver real value. But first things first, let’s just look at what Big Data actually means.

What Is Big Data?

Big Data basically refers to the fact that we can now collect and analyse data in ways that was simply impossible even a few years ago. There are two things that are fuelling this Big Data movement: the fact we have more data on anything and our improved ability to store and analyse any data.

More Data On Everything

Everything we do in our increasingly digitized world leaves a data trail. This means the amount of data available is literally exploding. We have created more data in the past two years than in the entire previous history of mankind. By 2020, it is predicted that about 1.7 megabytes of new data will be created every second, for every human being on the planet. This data is coming not just from the tens of millions of messages and emails we send each other every second via email, WhatsApp, Facebook, Twitter, etc. but also from the one trillion digital photos we take each year and the increasing amounts of video data we generate (every single minute we currently upload about 300 hours of new video to YouTube and we share almost three million videos on Facebook). On top of that, we have data from all the sensors we are now surrounded by. The latest smartphones have sensors to tell where we are (GPS), how fast we are moving (accelerometer), what the weather is like around us (barometer), what force we are using to press the touch screen (touch sensor) and much more. By 2020, we will have over six billion smartphones in the world – all full of sensors that collect data. But not only our phones are getting smart, we now have smart TVs, smart watches, smart meters, smart kettles, fridges, tennis rackets and even smart light bulbs. In fact, by 2020, we will have over 50 billion devices that are connected to the Internet. All this means that the amount of data and the variety of data (from sensor data, to text and video) in the world will grow to unimaginable levels.

Ability To Analyse Everything

All this Big Data is worth very little unless we are able to turn it into insights. In order to do that we need to capture and analyse the data. In the past, there were limitations to the amount of data that could be stored in databases – the more data there was, the slower the system became. This can now be overcome with new techniques that allow us to store and analyse data across different databases, in distributed locations, connected via networks. So-called distributed computing means huge amounts of data can be stored (in little bits across lots of databases) and analysed by sharing the analysis between different servers (each performing a small part of the analysis).

Google were instrumental in developing distributed computing technology, enabling them to search the Internet. Today, about 1000 computers are involved in answering a single search query, which takes no more than 0.2 seconds to complete. We currently search 3.5 billion times a day on Google alone.

Distributed computing tools such as Hadoop manage the storage and analysis of Big Data across connected databases and servers. What’s more, Big Data storage and analysis technology is now available to rent in a software-as-a-service (SAAS) model, which makes Big Data analytics accessible to anyone, even those with low budgets and limited IT support.

Finally, we are seeing amazing advancements in the way we can analyse data. Algorithms can now look at photos, identify who is on them and then search the Internet for other pictures of that person. Algorithms can now understand spoken words, translate them into written text and analyse this text for content, meaning and sentiment (e.g. are we saying nice things or not-so-nice things?). More and more advanced algorithms emerge every day to help us understand our world and predict the future. Couple all this with machine learning and artificial intelligence (the ability of algorithms to learn and make decisions independently) and you can hopefully see that the developments and opportunities here are very exciting and evolving very quickly.

Big Data Opportunities

With this book I wanted to showcase the current state of the art in Big Data and provide an overview of how companies and organizations across all different industries are using Big Data to deliver value in diverse areas. You will see I have covered areas including how retailers (both traditional bricks ’n’ mortar companies as well as online ones) use Big Data to predict trends and consumer behaviours, how governments are using Big Data to foil terrorist plots, even how a tiny family butcher or a zoo use Big Data to improve performance, as well as the use of Big Data in cities, telecoms, sports, gambling, fashion, manufacturing, research, motor racing, video gaming and everything in between.

Instead of putting their heads in the sand or getting lost in this startling new world of Big Data, the companies I have featured here have figured out smart ways to use data in order to deliver strategic value. In my previous book, Big Data: Using SMART Big Data, Analytics and Metrics to Make Better Decisions and Improve Performance (also published by Wiley), I go into more detail on how any company can figure out how to use Big Data to deliver value.

I am convinced that Big Data, unlike any other trend at the moment, will affect everyone and everything we do. You can read this book cover to cover for a complete overview of current Big Data use cases or you can use it as a reference book and dive in and out of the areas you find most interesting or are relevant to you or your clients. I hope you enjoy it!

1WALMARTHow Big Data Is Used To Drive Supermarket Performance

Background

Walmart are the largest retailer in the world and the world’s largest company by revenue, with over two million employees and 20,000 stores in 28 countries.

With operations on this scale it’s no surprise that they have long seen the value in data analytics. In 2004, when Hurricane Sandy hit the US, they found that unexpected insights could come to light when data was studied as a whole, rather than as isolated individual sets. Attempting to forecast demand for emergency supplies in the face of the approaching Hurricane Sandy, CIO Linda Dillman turned up some surprising statistics. As well as flashlights and emergency equipment, expected bad weather had led to an upsurge in sales of strawberry Pop Tarts in several other locations. Extra supplies of these were dispatched to stores in Hurricane Frances’s path in 2012, and sold extremely well.

Walmart have grown their Big Data and analytics department considerably since then, continuously staying on the cutting edge. In 2015, the company announced they were in the process of creating the world’s largest private data cloud, to enable the processing of 2.5 petabytes of information every hour.

What Problem Is Big Data Helping To Solve?

Supermarkets sell millions of products to millions of people every day. It’s a fiercely competitive industry which a large proportion of people living in the developed world count on to provide them with day-to-day essentials. Supermarkets compete not just on price but also on customer service and, vitally, convenience. Having the right products in the right place at the right time, so the right people can buy them, presents huge logistical problems. Products have to be efficiently priced to the cent, to stay competitive. And if customers find they can’t get everything they need under one roof, they will look elsewhere for somewhere to shop that is a better fit for their busy schedule.

How Is Big Data Used In Practice?

In 2011, with a growing awareness of how data could be used to understand their customers’ needs and provide them with the products they wanted to buy, Walmart established @WalmartLabs and their Fast Big Data Team to research and deploy new data-led initiatives across the business.

The culmination of this strategy was referred to as the Data Café – a state-of-the-art analytics hub at their Bentonville, Arkansas headquarters. At the Café, the analytics team can monitor 200 streams of internal and external data in real time, including a 40-petabyte database of all the sales transactions in the previous weeks.

Timely analysis of real-time data is seen as key to driving business performance – as Walmart Senior Statistical Analyst Naveen Peddamail tells me: “If you can’t get insights until you’ve analysed your sales for a week or a month, then you’ve lost sales within that time.

“Our goal is always to get information to our business partners as fast as we can, so they can take action and cut down the turnaround time. It is proactive and reactive analytics.”

Teams from any part of the business are invited to visit the Café with their data problems, and work with the analysts to devise a solution. There is also a system which monitors performance indicators across the company and triggers automated alerts when they hit a certain level – inviting the teams responsible for them to talk to the data team about possible solutions.

Peddamail gives an example of a grocery team struggling to understand why sales of a particular produce were unexpectedly declining. Once their data was in the hands of the Café analysts, it was established very quickly that the decline was directly attributable to a pricing error. The error was immediately rectified and sales recovered within days.

Sales across different stores in different geographical areas can also be monitored in real-time. One Halloween, Peddamail recalls, sales figures of novelty cookies were being monitored, when analysts saw that there were several locations where they weren’t selling at all. This enabled them to trigger an alert to the merchandizing teams responsible for those stores, who quickly realized that the products hadn’t even been put on the shelves. Not exactly a complex algorithm, but it wouldn’t have been possible without real-time analytics.

Another initiative is Walmart’s Social Genome Project, which monitors public social media conversations and attempts to predict what products people will buy based on their conversations. They also have the Shopycat service, which predicts how people’s shopping habits are influenced by their friends (using social media data again) and have developed their own search engine, named Polaris, to allow them to analyse search terms entered by customers on their websites.

What Were The Results?

Walmart tell me that the Data Café system has led to a reduction in the time it takes from a problem being spotted in the numbers to a solution being proposed from an average of two to three weeks down to around 20 minutes.

What Data Was Used?

The Data Café uses a constantly refreshed database consisting of 200 billion rows of transactional data – and that only represents the most recent few weeks of business!

On top of that it pulls in data from 200 other sources, including meteorological data, economic data, telecoms data, social media data, gas prices and a database of events taking place in the vicinity of Walmart stores.

What Are The Technical Details?

Walmart’s real-time transactional database consists of 40 petabytes of data. Huge though this volume of transactional data is, it only includes from the most recent weeks’ data, as this is where the value, as far as real-time analysis goes, is to be found. Data from across the chain’s stores, online divisions and corporate units are stored centrally on Hadoop (a distributed data storage and data management system).