AI for Retail E-Book

Francois Chaubard

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 446

Veröffentlichungsjahr: 2023

Beschreibung

The coming of the AI revolution in brick-and-mortar retail In AI for Retail: A Practical Guide to Modernize Your Retail Business with AI and Automation, Francois Chaubard, AI researcher and retail technology CEO, delivers a practical guide for integrating AI into your brick-and-mortar retail business. In the book, you'll learn how to make your business more efficient by automating inventory management, supply chain, front-end, merchandising, pricing, loss prevention, e-commerce processes, and more. The author takes you step by step from no AI Strategy at all to implementing a robust AI playbook that will permeate through your entire organization. In this book, you will learn: * How AI works, including key terminology and fundamental AI applications in retail * How AI can be applied to the major functions of retail with detailed P&L analysis of each application * How to implement an AI strategy across your entire business to double or even triple Free Cash Flow AI for Retail is the comprehensive, hands-on blueprint for AI adoption that retail managers, executives, founders, entrepreneurs, board members, and other business leaders have been waiting for.

Leseprobe

AI FOR RETAIL

A PRACTICAL GUIDE TO MODERNIZE YOUR RETAIL BUSINESS WITH AI AND AUTOMATION

FRANCOIS CHAUBARD

Copyright © 2023 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

The Wiley cover designer generated the type‐fill image in part with DALL•E 2, OpenAI's large‐scale image‐generation model. Upon generating a draft image, the Wiley designer reviewed, edited, and revised the image to their liking. Wiley is responsible for the image content of this book's cover.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750‐8400, fax (978) 750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748‐6011, fax (201) 748‐6008, or online at http://www.wiley.com/go/permission.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762‐2974, outside the United States at (317) 572‐3993 or fax (317) 572‐4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging‐in‐Publication Data is Available:

ISBN 9781394184699 (Cloth)

ISBN 9781394184705 (ePub)

ISBN 9781394184712 (ePDF)

Cover Design: Wiley

Cover Image: © Wiley/DALL•E 2/OpenAI

I dedicate this book to four incredible groups of people. First, my wife Taylor, who has been my rock through the highs and lows of this entrepreneurial journey. Second, to my team at Focal Systems, who inspire me every day as they lead the charge globally on applying AI to retail. Third, to all the mentors and advisors throughout my life who have taken me to school on retail, AI, how to build a successful business, and how to live a useful and fulfilling life that will continue to serve the world posthumously. And finally, to the Billy Beanes of retail who will have the courage and motivation to take these ideas the full distance to transform retail completely. I hope this book makes all of you proud.

Introduction: Why You Need This Book as a Retail Leader

In the summer of 1997, I was 10 and just entering the fifth grade on the outskirts of Manhattan. My father and I would play chess together most nights, and I enjoyed it very much. That year was perhaps the most exciting time in chess history since its invention 1,500 years ago. On May 11, 1997, not far from my home in downtown Manhattan, World Chess Champion Garry Kasparov faced off against IBM supercomputer Deep Blue (see Figure I.1). In a battle of man versus machine, the world gasped as the unthinkable occurred. Kasparov stood up and walked away in game six, thereby conceding the game to Deep Blue. AI won. This was an omen to the world of the transformation AI was soon to cause. This irrevocably peaked my interest in computers. After that year, my love for chess evolved into a passion for computer programming and AI.

FIGURE I.1 Gary Kasparov vs. Deep Blue, May 11, 1997.

Source: STAN HONDA / Getty Images.

A year later, my sixth grade teacher brought our entire class on a field trip to visit the New York Stock Exchange (NYSE). I remember vividly as I entered the main hall, I was nearly knocked over by the raucous ballet of men and women in lab coats belting “SELL, SELL, SELL,” or “BUY, BUY, BUY.” If you haven't experienced this yourself, Ferris Bueller's Day Off does a great job documenting this wild skirmish. In the midst of the chaos, I thought to myself, “How do they know what to buy? How much to offer? How do they know they are getting a good deal?” I asked our tour guide all of these questions.

The tour guide replied, “Experience, son; these brokers have been buying and selling stock for decades. They are the best in the world.” Naively thinking about Gary's stunning defeat, who was also the best in the world at his craft, I asked, “Do you think a computer could ever beat them like Gary Kasparov?” The tour guide smiled at me with confidence, and dismissed the notion. “Not a chance,” he said.

He was wrong.

Over the next 20 years, almost all of the trading volume on the NYSE would migrate to AI‐generated trades. Every day, AI algorithms ingest billions of rows of real‐world data, and every millisecond, they compute predictions on the future stock price to find arbitrage opportunities that humans could never find. In 2003, 15% of all trading volume on the NYSE was generated from a computer program. By 2016, 80% of all trading volume was generated from a computer program. Today this is closer to 95% (see Figure I.2).

If you visit the NYSE now, the raucous ballet is gone. Today, you could hear a pin drop on the trading floor (see Figure I.3). The traders were all laid off and replaced with quants, experts in analyzing and managing quantitative data, to create AI models to predict the right trades to make.

Why did this happen?

FIGURE I.2 Algorithmic vs. manual trading, 2004–2012.

FIGURE I.3 Evolution of trading pits over 16 years.

Source: xPACIFICA / Alamy Stock Photo.

The simple answer: Because it makes more money per unit risk compared to legacy methods.

The complicated answer: Computers are not emotional. They can't be bribed. They don't call out sick or come into work hungover. They can crunch huge amounts of data in a fraction of the time required by the fastest human. They are ruthlessly focused on maximizing an objective function, such as maximizing profits or minimizing risk or both, maximizing a joint objective function. They can access data from all around the world, and when something changes in the world, they make decisions on new trades that should be executed in milliseconds. They can see patterns and trends humans can't. Since finance is a sea of endless columns of data that are constantly changing and all columns have correlations with each other, one small change in the yield curve or change in the spot price of the USD to EUR exchange rate will have ripple effects throughout the entire economy, changing the price of every stock and every bond by a small amount. Only AI is qualified enough to play this game today.

Why did this happen in 2000?

This is a really important question. Why not earlier? Why not later? I will make this point again throughout the book, but the earlier you understand it, the better, so listen closely. There are three major prerequisites to get AI to work and achieve mass adoption. The first is access to accurate, real‐time data. Without accurate data, you will have garbage in, garbage out, and the AI will not work. The second is the right algorithms. Every domain is entirely different, and you frequently must create custom algorithms for every application, whether it is finance, retail, speech recognition, computer vision, or other fields. Rest assured these are all very different algorithms using the same principles and techniques of AI. In optimization, they call this the “no free lunch,” theorem meaning that “any two optimization algorithms are equivalent when their performance is averaged across all possible problems,” making the case for task‐specific AI. In the case of finance, in the mid‐90s, computer scientists had to create task‐specific AI, which did not know how to drive a car but definitely knew the stock market. Once both of these first two prerequisites were satisfied, someone had to be bold enough to put their money at risk and hope that it works! This is the last prerequisite for AI to revolutionize an industry. Let's discuss how these three unfolded in finance.

Data:

Michael Bloomberg launched the Bloomberg Terminal in the mid‐80s, which provided accurate, real‐time data. But not until the late 90s did these terminals and other input sources have the APIs for quants to write programs against. Without this, the AI did not have access to the necessary accurate, real‐time data to function properly.

Algorithms:

While there is a never‐ending invention of AI algorithms applied to finance, I will discuss the most famous one. In 1973, a few economists from MIT published a paper with a new class of algorithms to price options and corporate liabilities better than anyone else. It showed the financial world how the same math used in rocket science (stochastic calculus) could apply with great success to equity trading. No one took this paper seriously though.

Courage:

Courage has a lot to do with revolutionizing any industry, and finance was no different. After 15 years of waiting for someone to use these algorithms, the authors got tired of waiting. In 1994, they launched a hedge fund called Long Term Capital Management (LTCM) to exploit these market inefficiencies. These algorithms returned some of the highest returns the market has ever seen. However, four years later, a six sigma (very rare) event occurred that the algorithms could not model well, and the algorithms started buying and selling erratically, almost tanking the stock market, and certainly tanking LTCM. While LTCM is largely deemed a bust, it was hugely important for the adoption of AI trading in finance. “The first one through the wall always gets bloody” —Moneyball. Over the next 20 years following LTCMs crash, mathematicians took to Wall Street and made fortunes applying variants of the Black‐Scholes model to finance, such as Renaissance, DE Shaw, Two Sigma, Jane Street, Hudson River Trading, SIG, just to name a few of the most famous ones. 30 years later after the paper was published, they would be awarded the Nobel Prize for this contribution.

Every industry we know of has already been revolutionized by AI, or will be soon. As Marc Andreesen famously said: “Software is eating the world,” I now say, “AI is eating software.” And retail is next!

Why can't AI select SKUs better than humans? Why can't AI create labor schedules that maximize sales per labor hour better than humans? Why can't AI create better pricing strategies that maximize long‐term profits better than humans? Why can't AI write orders better than humans to shrink food waste and maximize sales?

Similar to the NYSE tour guide, I have heard retailers give me thousands of reasons why AI could never beat them, but I assure you, in 10 years AI will be doing so in almost all retail stores in the world, and it will happen faster than you think. The question is when, not if.

To answer “when,” let's first understand what is stopping retail from becoming automated now.

First, access to accurate, real‐time data. Retailers have no idea what is actually happening in their stores. What is in the backroom? What is on the sales floor? If you asked the CEO of any major retail chain to bet that Coke two‐liter is for sale right now in store 123 in the soda aisle, no CEO ever would. Retailers rely on very fuzzy, inaccurate, stale data to make the most important decisions. Once retailers have access to real‐time POS feeds and shelf information of the sales floors and backrooms by some type of sensor, then they will be able to fully apply AI to their stores.

Second, retailers do not have the right algorithms. Retailers rest on extremely simple algorithms to run their stores. Orders are based on knowingly inaccurate store inventory data and forecasts. Staffing budgets are just a percentage of sales, not driven from the bottom up from what work needs to be done in the store. These are all simple, four function math models that result in suboptimality all over the place. A new “theory for retail” needs to be uncovered similar to what Scholes, Black, and Merton did for finance in the 90s. I am hopeful that this book will serve as the template for those algorithms.

Last, but not least, someone has to do it. Retailers are more conservative than most industries. In a 3% net margin environment, a risky move could kill the company. So it's understandable why most retailers have not adopted almost any AI yet. But Amazon is not most retailers, and they are doing it now in every suborg at scale. Dynamic merchandising, dynamic pricing algorithms, etc. Yes, the first one through the wall gets bloody, and Amazon Go in my opinion is an example of that, but they are showing the world what the art of the possible is and how to do it and how not to do it as well. Walmart is taking those learnings and converting them into better implementations of AI. We are in the early phase of this adoption cycle, but we are about to hit a huge uptick. Yet many retailers are not equipped for this new reality.

As a retail leader you can no longer ignore AI's tremendous role in the future of retail. You spend 13% of sales on labor. And likely 0% of sales on AI. Amazon will spend 4% of sales on labor and 3% on AI and will make 6% more net margin than you doing so. Your P&L can look like that in the future with even higher sales.

This is why every retail leader needs a strong AI strategy, just like they have historically needed a strong labor strategy. And it has to stem from a deep understanding of how AI works, what it can do, and what it can't. Most retailers I know have either no AI strategy or worse, they have a terrible one, i.e., we are going to test a bunch of AI solutions. Aimlessness is a vice. That's not a strategy. That's unintelligent meandering. This book is dedicated to helping you not commit either of these sins. But instead to give you a foundational understanding of what AI is, how it works, how powerful it can be on each aspect of your business, and how to use that knowledge to devise and implement an AI strategy that will double or triple your free cash flow (FCF) for your business in under three years. And if not you, then your competitor will, and you may end up on a growing list of retailers who have already gone bankrupt.

Before we do a deep dive into how retail will be transformed by AI, I want to provide some leaves on the tree of how AI has automated other industries in the past 20 years. These examples will serve as a foundation for my prediction of how AI will transform retail.

How to Use This Book

My goal in writing this book is to serve as your end‐to‐end guide as you digitally transform your team, your culture, your systems, and in the end your business to become a powerful AI‐driven business. I want you to make your AI prowess something your competitors envy in you, rather than the other way around.

I recognize that the retail sector is very conservative, full of tradition and old‐school thinking, just like baseball. And similar to baseball, my guess is that of the three prerequisites, the third will take longer than it should. I wrote this book for the Billy Beane of retail. The ideas in this book are a massive departure from the status quo, and implementing them will be met with resistance and disbelief by the old guards at every turn. If you have enough motivation to be reading this book, perhaps you have enough motivation to be the Billy Beane of retail. I know someone will eventually fully implement these ideas; I just hope this book helps you along the way.

While reading this book, I recommend you snap out of a “skim quickly” mindset and, instead, come in with a learning mindset. When you get to a paragraph that has you stumped or that does not fit with your understanding of retail, slow down and meditate on it a bit. Similar to Alpha Go's Move 37, AI will find new ways to “play the game” of retail that will challenge the status quo. To understand the premise of this book, you may have to forget a lot about what you have learned over your career in retail and relearn a new way of thinking. This is the fresh slate, tabula rasa mindset that Amazon has as they are currently entering into B&M retail, unburdened by the current way of doing things, and you too could share this same benefit if you put your mind to it.

I prefer that you use this book in three ways. First, I want you to read it through all the way once to have a general understanding of the subject matter. Second, I want you to give it to your team underneath you so they can speak the same language as you when discussing AI topics. If you are not a retail leader, give it to your boss so they can understand what you now know. Only when your team shares the same context as you can you really lead them through this critical change. Then finally, once you have begun to transform your retail business with AI, I want you and your team to use this book as a reference to guide you as you make the journey from a legacy mindset and systems to an AI‐based mindset and systems.

Throughout this book, I specifically address leaders in the retail industry. If you are an executive, you may be asking, “Why do I need to know about the fundamentals of AI? Shouldn't that responsibility lie with the store associates who will be interfacing with the technology on a daily basis? Or can I offload this understanding to my IT or innovation teams?”

Or if you are not a retail executive but you work in retail, do you trust in your current leadership to navigate your organization from legacy systems to an AI‐driven organization without learning the subject matter themselves?

In either case, no.

This transformation has to be top‐down. If it comes from the bottom‐up, it will certainly fail as different departments struggle with disparate systems and a lack of overall vision or incentive to push such a major change through. As a retail leader, the origin of this transformation must begin with you. As stated in the Harvard Business Review article “Building the AI‐Powered Organization,” you have to “walk the talk” if you want this to stick. Role modeling is essential. If you do not take it seriously, neither will they.

Additionally, a fundamental rule of leadership is that you cannot lead what you do not understand. Could you imagine an AI engineer running a grocery store without ever working in grocery? There is no way that would work. In a similar way, however, a retail executive trying to run an AI‐powered retail store with absolutely no understanding of AI cannot work either. Every retail leader will require a solid understanding of the fundamentals of both retail and AI, and that is what I aim to provide in this book.

So let's discuss goals. By the end of this book, my goal is that you (or your retail leader) will develop an intuition for what AI is, how it works, its limitations, what it can do for them, and finally how to implement it.

I have split this book into four sections.

The second section is a very quick overview of AI theory. This is mostly for you to be familiar with common terms in AI so you can speak intelligently with your team about AI or with an AI vendor that might have a great product or might be selling snake oil. By the end of this section, you will be able to ask the right questions to separate the wheat from the chaff.

The third section is about applications of AI in retail. Of the many tasks a retailer must perform, there is no shortage of approaches to try to solve them. We will go deep into the pros and cons of all of them so you can skip years and millions of wasted dollars in POC‐ing technology that won't work for you, accelerating your organization by leaps and bounds toward the automation of your business. I will provide a huge list of AI use cases that may be applicable to your stores.

The last section is all about implementation. In this section, we will discuss how to create an AI strategy and road map for your business to execute against for years to come. We will discuss key roles that you need to fill, systems and norms to put in place, and ways to evaluate different opportunities and AI solutions. We will discuss when to partner and when to build. And most importantly, we will describe how to create an AI‐driven culture that will become your key advantage over your competitors.

Now, let's jump into the world of AI. And let's start at the beginning with a quick primer on AI theory.

SECTION 1Introduction to the AI Revolution

CHAPTER 1How AI Has Revolutionized Many Other Industries over the Last 20 Years

“History never repeats itself, but it does often rhyme.”

—Mark Twain

To understand how AI will transform retail, it's important to understand how AI has already transformed, or is transforming now, countless other industries. I've included some of my favorite examples for us to have a clearer understanding of what is going to happen to retail.

Just like in finance, the three prerequisites to look out for in all of these examples are:

There needed to be a step function in availability of accurate, real‐time data;

There needed to be a step function in task‐specific algorithms;

Someone had to have the courage to prove that this new model will work.

Once all three of these have occurred, the floodgates open, and AI infiltrates the entire industry with a ferocious pace. Let's dig into some examples.

Advertising Revolutionized by AI in 2000

Marketers have always been trying to get us to buy their product, go to their stores, or change our behavior in some way. Classic marketing strategy goes this way. First, identify your target markets/audience/personas, and then create campaigns per segment with ad placement to target those personas to maximize “conversion” rates per dollar of ad spend. For example, if you were selling luxury handbags, you would likely be targeting a core demographic of perhaps: only women, between 30 and 70, and of a higher economic status. We would then try to target that group as much as possible with every dollar spent in advertising.

Before 2000, ad placement employed a similar strategy to a sawed‐off shotgun, broadly spraying the same ad to the masses, advertising pantyhose to men, Big Macs to vegans, and brake pads to people who don't own cars. This was true in print, TV, billboards, and even online, where marketers knew 90% of their ad spend would be wasted on the wrong people.

Then came the concept of “personalization.” Before AI, personalization was a marketer's pipedream. For personalization to work, advertisers needed to know many things about you. But before 2000, there was no data set in the world of what you, the advertisee, liked, didn't like, where you lived, what car you drove, what your political beliefs were, how old you were, what age, sex, religion you were, etc.

Until Google. In 1998, Google launched the best search engine of all time, an algorithmic advancement called PageRank. Because of this improvement in user experience and AI prowess, customers all around the world freely handed Google their personal data, data that Yahoo, MSN, AskJeeves, or any of the other search engine competitors could not gather, infer, or interpret to a level accurate enough to be able to target. Two years later, Google had access to accurate, real‐time information on hundreds of millions of people. With Google Maps, Chrome, and the DoubleClick acquisition, they had more data on you than the US government. They knew where you were right now, where you wanted to go, and what you wanted to buy. Additionally, they knew it sometimes quicker than you did! With AdWords/AdSense, they mined that data and provided marketers the ability to turn their sawed‐off shotgun into a sniper rifle, which revolutionized advertising forever. For the first time in history, marketers could target Vietnamese American males between 20 and 22 in Des Moines searching for ice cream shops right now and hit them with an ad in a millisecond, which dropped the cost‐per‐click (CPC), or the cost to get someone to click on my ice cream ad, by 100 to 1,000 times.

Each time you search on Google or load a page powered by AdSense, Google is figuring out which ad to show using AI models that are trying to maximize the probability that you will click on that ad and convert. The formulation of this problem is known as the “multi‐armed bandit” problem, which goes like this: Imagine you are in Vegas, and you have access to millions of slot machines, each one pays out some reward (Ri) at some probability (Pi). But you do not know these numbers for each slot machine before you start pulling, so you have to “explore” and then “exploit.” You start with one slot machine, see how often it pays out, explore another sometime later, pull that a few times, and see how often that one pays out. This is repeated until you have explored enough slot machines and maybe you found a few that pay out really well, so instead of exploring more and more, you slow it down a bit, and start exploiting the slot machines you know pay out well.

Similarly, with advertising, you start with a new ad, see how often people similar to you click on it, then see how well other ads work on you. This is repeated until they figure out what you are likely to click on and what you are not, customizing every single page you see on the web to maximize your click‐through rate (CTR), or percent of the time a user clicks on your ad. One of the most popular techniques to solve this is a class of AI algorithms called collaborative filtering.

Since then, Facebook, Instagram, Snapchat, Yelp, Amazon, TikTok, YouTube, and many other tech platforms have employed similar recommendation systems to grow their advertising revenue.

This innovation dropped the cost of advertising so low that only tech companies can really play in the advertising business anymore, forcing many classical ad‐driven industries such as print newspapers, billboards, radio, and television companies into turmoil and some into bankruptcy giving rise to the modern duopoly that is Facebook and Google.

For more information on this, watch the documentary The Social Dilemma.

Baseball Revolutionized by AI in 2002

Moneyball is one of my favorite movies. It's a true story of how AI transformed Major League Baseball (MLB) in 2002 irrevocably. It's based on Oakland A's General Manager Billy Beane (see Figure 1.1), who lost all of his best players in 2001. The team was likely to rank last place in the league.

FIGURE 1.1 Billy Beane; Oakland Coliseum, 1989.

Source: Silent Sensei / Wikimedia Commons / CC BY 2.0.

After losing a specific trade to another team, he noticed the opposing GM continuously consulting with a young economics major from Yale. Beane poached him to aggregate player statistics from the major and minor leagues and deploy AI algorithms on this data to select and manage a winning team on a shoestring budget. This was the first time AI was used to scout players and manage a baseball team. Beane's AI scouted team went on to break a number of baseball records, best summarized by Boston Red Sox GM in the last scene in the movie:

“For $41m you built a playoff team. You lost Damon, Giambi, Isringhausen, Pena, and you won more games without them than you did with them. You won the exact same number of games as the Yankees, but the Yankees spent $1.4 million per win and you paid $260,000. I know you're taking it in the teeth out there but the first guy through the wall, he always gets bloody…always. This is threatening not just their way of doing business, buddy, but really it's threatening their livelihood, it's threatening their jobs, and every time that happens whether it's a government or way of doing business or whatever it is, the people who are holding the reins, who have their hands on the switch, they go batshit crazy. I mean anybody who's not tearing their team down right now, and rebuilding it using your model, they're dinosaurs. They will be sitting on their ass on the sofa in October watching the Boston Red Sox win the World Series.”

The Red Sox owner was right. With AI, the Boston Red Sox won the World Series in 2004, for the first time since 1918. Over the next four years, every single MLB team hired swarms of statisticians and data scientists to mimic what Billy Beane and the Red Sox did.

The three major step functions that occurred to enable this AI revolution were, first, the “Society for American Baseball Research” (SABR) was established in 1971, which began recording and publishing all player statistics and metrics (accurate, and real‐time data). Second, in 1978, a guy named George William James (Bill James), an American baseball writer, historian, and statistician, started putting out an annual booklet called “The Bill James Baseball Abstract,” which described a new algorithm for running a baseball team called “sabermetrics” in reference to SABR. And third, in 2002, a guy named Billy Beane was crazy and desperate enough to actually try it, risking his entire career on the idea. Because of these three events, baseball will never be the same.

Computer Vision Revolutionized by AI in 2012

This is the most important advancement in AI that I will cover in this book. In 2006, my Computer Vision Lab at Stanford pulled together a huge data set of images downloaded from the Internet and labeled them. This data set is now famously called ImageNet. It is a huge data set of 22 million images of cars, dogs, cats, stop signs, etc. Since 2010, Stanford has used this data set for a global computer vision competition (called ILSVRC, ImageNet Large Scale Visual Recognition Competition) to try to provide the AI community a clear measure of how strong our AI models are in relation to each other. Almost all computer vision labs in the world submit their best AI model each year to try to win first prize. Winning this competition is a huge deal for computer vision researchers.

The grading of the competition goes like this. There are 1,000 possible “classes” or types of objects you want your AI to accurately identify, such as cats, dogs, stop signs, etc. Your submitted AI model will be given a few thousand images for which the correct answer is hidden, and the AI has to predict which of the 1,000 classes are present in each image. Stanford knows the true answers and then computes each submissions scores against the real answers and reports the results.

In 2010, the @1 Accuracy of the winning solution was around 40%. This means if the AI gets to make only one guess out of 1,000 possible classes, it will only get the right class 40% of the time, and 60% of the time would guess the wrong answer. This is really poor performance and would not be usable by most applications. Take self‐driving cars for example. I certainly would not get in a self‐driving car that would miss 60% of stop signs, would you?

In 2012, however, three authors who are now famous in the AI community (Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton out of the University of Toronto) published one of the most important papers in the world describing their model now known as AlexNet. They won the ILSVRC 2012 competition by a huge margin. They brought the state of the art @1 Accuracy from below 26% to 63.3%, an increase of 37%! This was almost too good to be true. Most labs thought they cheated. They did not. Their AI was just that superior to what any other lab was doing. With this tremendous success, these three gave birth to the AI renaissance we are experiencing today. Using their technique, inventors have been able to produce products such as Alexa, Siri, self‐driving cars, and more. I will explain this model in great detail in the theory section of this book, but to provide some context now, they were the first team to (1) ingest huge amounts of data into their AI models leveraging GPUs to make the computation very fast, (2) feed it into a “very deep” model (nowadays this model is puny but for its time was very “deep”), and (3) take Geoffrey Hinton's 1980 paper called “Back Propagation” as a way to optimize this very deep model with stochastic gradient descent (SGD). This is now the framework for all modern AI. By doing so, the AI model was able to learn filters on images that mimic what neuroscientists have discovered in the occipital lobe in our brains where each box in Figure 1.2 represents a “detector” that the AI is looking for (as proven by neuroscientists David Hubel and Torsten Wiesel in 1964).

While this was a huge step forward, to level‐set, 63% @1 Accuracy is not that good. That means that 37% of the time the AI gets it wrong. If this were a self‐driving car looking for stop signs, 37% of the time it would blow right through them. So not usable. But with this technique, they showed the AI community how to get there. Now we have continued to increase the size and sophistication of the models, and we have pushed the art of the possible on performance (see Figure 1.3).

Today, the state‐of‐the‐art accuracy on ILSVRC2012 is 91% @1 Accuracy. This is almost another 30% points of improvement. But still, 9% of the time the AI makes an error. To be fair, human‐level performance on this task is about 85%, so this is much better than humans, but still not perfect. To understand how hard the last 9%