22,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Geisteswissenschaft
- Sprache: Englisch
Implement AI and big data at your organization using principles from behavioral economics
In Behavioral AI: Unleash Decision Making with Data, behavioral economist Dr. Rogayeh Tabrizi delivers an intuitive roadmap to help organizations disentangle the complexity of their data to create tangible and lasting value. The book explains how to balance the multiple disciplines that power AI and behavioral economics using a combination of the right questions and insightful problem solving.
You'll learn why intellectual diversity and combining subject matter experts in psychology, behavior, economics, physics, computer science, and engineering is essential to creating advanced AI solutions. You'll also discover:
- How behavioral economics principles influence data models and governance architectures and make digital transformation processes more efficient and effective
- Discussions of the most important barriers to value in typical big data and AI projects and how to bring them down
- The most effective methodology to help shorten the long, wasteful process of “boiling the ocean of data”
An exciting and essential resource for managers, executives, board members, and other business leaders engaged or interested in harnessing the power of artificial intelligence and big data, Behavioral AI will also benefit data and machine learning professionals.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 300
Veröffentlichungsjahr: 2025
Ähnliche
Table of Contents
Cover
Table of Contents
Title Page
Copyright
Dedication
Preface
CHAPTER 1: Magic Happens at the Intersections
Asking the Right Questions: Data, Intuition, and Strategy
Simplifying the Complexity
Connecting the Dots
Uncovering Hidden Patterns: Models and Algorithms in Action
Decoding Consumer Behavior: The Interplay of Psychology and Economics
Empowering Behavioral Economics: The Synergy of Data Analytics, ML, and AI
Crafting a Customer‐Centric Paradigm: The Fusion of Technology and Behavioral Insights
CHAPTER 2: It Is All Connected
History and Origins of Behavioral Economics
Early Days
Entering Mainstream Economics
Current Research and Practical Applications
Back to the Beginning
Psychology of Decision‐Making
Nudging
Experimentation
CHAPTER 3: Minimal Data, Maximal ImpactFrom Big Data to Minimum Viable Data
How Much Data Are We Talking About? Lots and Lots
You Do Not Need a Lot of Data to Get Started, You Need the MVD
Asking the Right Questions, Again!
Synthetic Data: What It Is and What It Isn't
Survey Data to the Rescue
CHAPTER 4: Building Intelligence
Classical AI
ML
Deep Learning
Generative AI
Machine Intelligence and Biologically Inspired Models
CHAPTER 5: Real‐World Impact
Unleashing the Full Potential of AI: Beyond the Hype
Rethinking Segmentation: Beyond Demographics and Life Stages
Predicting Intent and Mapping Customer Journeys
The Power and Nuances of Recommendation Models
Leveraging Propensity Models for Targeted Campaigns
Personalized Pricing: Influencing Behaviors and Financial Outcomes
Forecasting: Understanding the Dynamics of Demand
The Power of Forecasting and Optimization
Conclusion
CHAPTER 6: Decoding Complexity
Only a Wet Baby Likes Change: Loss Aversion + Status Quo Bias
It Gets Better! Commitment Device, Peer Effect, and Sunk Cost Fallacy
Conclusion
CHAPTER 7: Unlocking Scale
Enablers of Success
Communication and Intellectual Diversity
Building Trust, Experimentation, and Adoption
Interpretation Layers
The Power of Experimentation
Measuring ROI Through Experimentation
Conclusion
Epilogue
Notes
Preface
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Additional Reading
Additional Reading on Applications of Classical Machine Learning
Additional Reading on Applications of Machine Learning
Additional Reading on Deep Learning Models
Additional Reading on Generative AI
Additional Reading on Deep Learning and Biologically Inspired Models
Bibliography
Acknowledgments
About the Author
Index
End User License Agreement
List of Illustrations
Chapter 4
FIGURE 4.1 Overview of AI and ML: This diagram illustrates the hierarchical ...
FIGURE 4.2 Time line of key milestones in artificial intelligence: This time...
Guide
Cover
Table of Contents
Title Page
Copyright
Dedication
Preface
Begin Reading
Epilogue
Notes
Additional Reading
Bibliography
Acknowledgments
About the Author
Index
End User License Agreement
Pages
iii
iv
v
xi
xii
xiii
xiv
xv
xvi
xvii
xviii
xix
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
179
180
181
182
183
184
185
186
187
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
205
206
207
208
209
210
211
212
213
214
215
216
217
ROGAYEH TABRIZI, PhD
BEHAVIORAL AI
UNLEASH DECISION MAKING WITH DATA
Copyright © 2025 by John Wiley & Sons, Inc. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750‐8400, fax (978) 750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748‐6011, fax (201) 748‐6008, or online at http://www.wiley.com/go/permission.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762‐2974, outside the United States at (317) 572‐3993 or fax (317) 572‐4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.
Library of Congress Cataloging‐in‐Publication Data is Available:
ISBN 9781394196869 (Cloth)
ISBN 9781394196852 (ePub)
ISBN 9781394196845 (ePDF)
Cover Design: Wiley
Cover Images: © pressureUA /Getty Images and Wiley
To my parents, who planted the seed of knowledge in my mind and nurtured it, you are my universe and constant source of inspiration and strength.
To my team and all who cheered me on during this incredible journey, and who saw in me what I couldn't see in myself.
To all my teachers, especially the great ones, who sparked my curiosity and ignited my passion.
To the challenges that made me stronger.
Preface
“I have no special talents. I am only passionately curious.”
– Albert Einstein
Much of this book has been written during plane rides to and from client meetings and conferences and is filled with stories, examples, reflections, and considerations drawn from countless conversations with clients, executives, and practitioners.
In these pages, I aim to take you on a journey through the realms of data science, behavioral economics, and organizational complexity. Together, we'll explore how to harness the power of predictive models and AI to uncover hidden patterns in data, drive informed decision‐making, and ultimately create value within your organization. By sharing real‐world examples and case studies from various industries, my goal is to provide you with practical insights and strategies that you can readily apply to your own work.
We will discover how to effectively combine data‐driven insights with an understanding of human behavior to overcome challenges and drive change within organizations. We will explore concepts such as game theory, cognitive biases, and behavioral economics, and how they can be combined with machine learning and AI models to enhance decision‐making processes and improve customer experiences.
Navigating organizational complexities and overcoming silos can make it challenging to identify the right datasets, build high‐quality models, and implement them effectively to achieve tangible results. Through the numerous stories and experiences shared in this book, I aim to provide you with actionable strategies to harness the power of your data. Additionally, you will gain a deeper understanding of the significance of addressing cognitive biases and fostering a collaborative, transparent environment to ensure the success of your initiatives.
Ultimately, this book aims to equip the practitioners and executives with the tools and knowledge needed for AI and behavioral economics to navigate the complexities of modern organizations, make data‐driven decisions, and lead your team to success. Through the lessons and stories shared, I hope to inspire you to embrace innovation, challenge the status quo, and unlock the full potential of your organization.
My own story starts on a plane ride from Tehran to Vancouver. I grew up in Tabriz, a city in the northwest of Iran, at the intersection of many cultures: Turkish, Azari, Persian, Kurdish, among others. As a young kid, due to my father's job, we traveled and lived in many different cities where we often didn't speak the native language, and frequently moved in the middle of the school year.
Being competitive with a natural drive for excellence, combined with my love for mathematics, made me pay extra attention – not just to my teachers, but to everything around me – to make sense of things and stay a top student. I remember paying attention to what drove my classmates, recognizing similarities and differences across cultures and language barriers. Without realizing it, I was being trained in behavioral sciences, but it took me almost 20 years to connect the dots.
I arrived in Vancouver when I was 21 to pursue my master's in physics, specifically in string theory. I had fallen in love with a particular area of mathematics called algebra, group theory, and algebraic topology, which lends itself beautifully to string theory and particle physics. This might sound strange, but the beauty, elegance, and power that this area of mathematics has to simplify the complexities of our physical world have guided my thinking and approach all these years later in the work I do with Fortune 100 companies.
Shortly after starting my master's, I switched to particle physics. This was partly because, no matter how much I love math and theoretical physics, I need to see the physical manifestation of these fascinating theories – I need to see their impact or reality in practice. I want to experiment and see what happens. I call this my impact bug. The timing was also perfect: CERN was up and running, almost! I was one of the last groups of people who got to see the ATLAS detector open. It was a moment that I will never forget, one that changed my life and my relationship with the word impossible. I remember thinking that if we can build this machine that helps us understand what happened 10−16 seconds after the big bang, we can do anything!
I sometimes jokingly say, “Impossible doesn't quite occur to me!” and it's thanks to that moment. I also will never forget the first time I saw the Tier 3 facilities at CERN: a computer farm so vast that you couldn't see the end of it. That was my introduction to big data. The first real data we received after the initial collisions was 7 terabytes (TB), and it didn't even occur to me that it was big. It was just what it was! We had to figure out how to parallelize our “jobs” over however many cores and pray to God that we had caught all the bugs in our codes before submitting them!
There were many other lucky moments for me during my time at CERN. Getting to know the senior management at CERN was one, and having the opportunity to be the youngest member of the organizing committee of the First African School of Physics was another. I got to see the inner workings of one of the most sophisticated and complex scientific collaborations: 10 000 scientists from more than 100 countries coming together for 40 years to make a Nobel Prize–winning discovery. Only years later did I realize how I was being trained in systems thinking and life at 32° Fahrenheit, as described in Loonshots by Safi Bahcall,1 by some of the best scientists in the world.
In Loonshots, Bahcall talks about phase transitions, where small changes in conditions can lead to dramatic shifts in behavior. This concept was evident in the way CERN managed to align diverse talents and resources to achieve groundbreaking discoveries. I also got to see how some of these very same physicists came together and asked a simple question, leading to a decade‐long journey: How can we build foundational capabilities in fundamental physics in a continent, Africa, with massive potential? With careful planning, building relationships, and networks of the right people and institutions, I witnessed more than 700 graduate students participate in a three‐week summer school in fundamental physics over 10 years. More than 70% of them ended up doing their PhDs and postdocs in Europe and North America, with 35% returning to their home countries. I got to see how a movement starts and spreads across a continent, much like a phase transition where initial efforts create a ripple effect, leading to significant and sustained impact.
All this and more led me to decide to study economics for my PhD. I wanted to delve into development economics. The reason was simple: How is it that we spent $20 billion and 40 years to build a machine that can answer what happened 10–16 seconds after the big bang, but spent $3 trillion over 40 years in international aid in Africa and still can't keep a polio vaccine cold enough to reach a village? I wanted to understand what we can do better, what we can do differently. The impact bug had hit me yet again.
Determined to pursue the economics of development, I started my PhD only to fall in love with game theory in the first semester. Game theory is the study of why people do what they do, the way they do it. It was the second most beautiful thing I had ever encountered after the standard model of particle physics. I was particularly fascinated by social and economic networks and how games of incomplete and asymmetric information unfold within these networks. I wanted to understand what drives behavior when individuals are influenced by those around them, or why a particular technology takes off rapidly in one society but never gains traction in another. What makes some people more influential than others, and what are the underlying conditions that lead to positive spillovers or strategic complementarities?
These questions led me to explore the interconnectedness of human behavior and economic outcomes, as beautifully articulated in Social and Economic Networks by Matthew Jackson,2 one of my mentors and advisors. His work provided profound insights into how networks form, evolve, and influence economic activity, shaping my understanding of the complex web of interactions that drive development and innovation.
As much as game theory was different from particle physics – as if I only wished people would behave like atoms – I was still a model generation machine. It took me nearly three years to realize that I was undergoing a paradigm shift, trying to understand how an economist thinks and why it differs from the mindset of a physicist. Every week, I arrived at my supervisor's office with my shiny new model. After patiently listening to me explain it, he'd invariably ask, “But what is the question?!” or “What is the intuition behind the model?” I painfully anticipated this question every time. It wasn't that I didn't have an answer; I just didn't understand the question! I had a model with a clear set of assumptions explaining a particular dynamic, equilibrium state, or evolution in the behavior of my “agents.” Very physicist of me!
It took me three years to understand what he meant because, slowly, I was developing an intuition. I was learning, ever so subtly, to question the questions, to listen, and to search for the underlying, hidden assumptions that I was making without quite realizing it. I was also learning to identify them in others' arguments and papers. I began to make connections between seemingly unrelated, sometimes contradictory, facts. It was starting to become natural to me, much like how I played around in my head with group theory and algebra. I was able to tap into an area of math that felt natural to me and use the art of asking questions to peel back the layers of the onion one by one, find the right question, and backward engineer from there. It was first principle thinking in action.
This transformation was crucial, as it enabled me to bridge the gap between abstract models and real‐world applications. It wasn't just about creating elegant models anymore; it was about ensuring these models had a solid foundation in reality and could provide genuine insights into human behavior and economic dynamics. I also realized that in physics, I was trained to optimize: to find the best, most efficient way to solve a given problem. In economics, however, the question is never given. Our job is to first find the right question and then solve it. It was magical to sit at the intersection of these disciplines and seamlessly – much to my advisors' surprise – go back and forth between them. This skill has become my most important asset in solving complex problems today. It's a skill I see sorely lacking in many data scientists trained in physics, computer science, or engineering.
But transitioning from academia to the real world wasn't simple, especially with my background as a theoretical physicist, an experimental physicist, and then an economist. The flexibility to apply different methods and tap into tools across multiple disciplines, coupled with the ability to think more fluidly and flexibly, plus many years spent in university, made me a strange outsider when I decided to leave academia and not pursue a tenure‐track position.
Yet, this multidisciplinary training became my superpower. It enabled me to approach problems from unique angles, ask questions others might overlook, and combine insights from various fields to create innovative solutions. When I decided to move beyond the academic world, I brought with me a tool kit rich in diverse methods and a mindset geared toward continuous learning and adaptation. This journey has shaped my ability to navigate and thrive in the complex landscapes of today's data‐driven world.
The first year was the hardest: like many with ties to universities in Silicon Valley, such as Stanford, I started my career at a startup in the Valley. Shortly after, I moved back to Vancouver to work as a data science specialist for a unicorn in Canada. I was the most senior data scientist in the company, and my life couldn't have been more different from walking the hallways of the economics department at Stanford or sitting in the cafeteria at CERN. Nothing in my education had prepared me for the real world, especially for talking to “normal people” (as I jokingly used to say). No one understood what I said, why I suggested a particular method or approach, or why I was asking for more data. Anything I did, and how fast I did it, felt like magic to my colleagues.
So, I had a brilliant idea: leave a well‐paying job and do a bit of consulting to figure out what I wanted to do when I grew up. I asked a few friends to introduce me to executives in different industries. That's how Theory+Practice was born, and an unexpected phenomenon occurred almost immediately: The first client I signed was a billion‐dollar retailer, the second was a $2 billion e‐commerce company, and the third was the world's largest logistics and express transportation company – all within the first six months of the company. The second year was even more surprising, as I signed with the largest retailer in the world, followed by several other major Fortune 500 retailers and financial institutions, and then the largest consumer packaged goods and supply chain companies in the world.
I had experienced very steep learning curves when transitioning from theoretical physics to experimental physics, and again when moving from a master's in particle physics to a PhD in economics without any background in economics. But this wasn't a learning curve – it was a learning wall!
I was fascinated by enterprise and all the incredible complexity of these environments. Looking at problems from the freshest, most outsider perspective possible, I believe this was one of the key reasons behind the traction we gained. I couldn't understand why we needed to wait for all the data to be cleaned and organized before starting to answer the most important and strategic questions. I remember jokingly telling executives that if we had waited to clean 20 TB of data at CERN, there would be no Nobel Prize, no discovery! I wanted data and a lot of it, and I wasn't going to wait. That's how – paradoxically – I started using the phrase minimum viable data. I knew that by applying proper methodologies from the intersection of different disciplines, we could build high‐quality models to test various hypotheses.
I also attracted an incredibly multidisciplinary team of physicists, economists, engineers, and computer scientists whose curiosity and hard work pushed the boundaries of what everyone around us claimed to be impossible. We loved tackling seemingly impossible problems.
I have dedicated most of my adult life to studying ways to detect patterns in vast quantities of data, whether it was searching for the signatures of the Higgs boson among oceans of other particles or signals for the underlying factors driving particular behaviors. Being trained at the intersection of physics, economics, and behavioral sciences is humbling, yet it has enabled me to cross‐pollinate and realize that there are many ways behavioral science can benefit from methodologies used in physics and computer science, and vice versa.
By illustrating the practical applications and lessons learned in the following chapters, I aim to show how economic intuition and behavioral economic methods can become incredibly useful tools. These approaches not only help to decode the complex patterns hidden within vast datasets but also drive meaningful and impactful decisions. Through this exploration, I hope to provide valuable insights into how we can leverage these interdisciplinary methods to solve some of the most challenging problems in today's data‐driven world.
CHAPTER 1Magic Happens at the Intersections
“I would rather have questions that cannot be answered than answers that cannot be questioned.”
– Richard P. Feynman
In September 2022, I found myself under the spotlight at a renowned retail conference, the first in‐person conference since COVID‐19. The pandemic had accelerated the digital and data transformation efforts of many large enterprises by more than five years. Big data, artificial intelligence (AI), and machine learning (ML) were among some of the most important sessions because many executives believed that they could use these advanced methodologies and predictive models to understand what their customers' wants and needs were, and how they could serve them better by providing more personalized products, prices, and marketing. However, they were wondering how they can use vast amounts of data to predict how customers' demands have changed in order to optimize their productions and shipments to minimize excess inventory as well as out‐of‐stock challenges.
I shared how so many of my conversations with executives across finance, insurance, retail, and consumer packaged goods all start with statements like, “We have so much data and we do not know what to do with it.” I gave examples of using AI and ML models from our recent projects during the pandemic. Then, I addressed the elephant in the room: consumer preferences have changed, and it's more crucial than ever not to rely solely on deterministic models that fail to capture these dynamics.
I paused on a slide with the title “Magic Happens at the Intersections of Data, AI, and Behavioral Economics” to ensure the message was clear. Each of these topics alone is complex and the subject of many books and discussions. Individuals earn their PhDs in AI or behavioral economics after years of study and research. And here I had 45 minutes to talk about the magic at the intersection of these disciplines!
AI and ML encompass the creation of sophisticated models and algorithms designed to execute tasks traditionally requiring human intelligence. These tasks span a wide range of capabilities, including reasoning, learning, perception, problem‐solving, language comprehension, and even creativity. At their core, these models are driven by data, learning to identify patterns and make informed decisions with minimal human intervention. One of the most remarkable aspects of AI and ML is their ability to adapt to new circumstances and continuously improve over time. This adaptability not only enhances their accuracy and efficiency but also opens the door to innovative applications across various industries, transforming how we approach complex problems and decision‐making processes.
Behavioral economics, however, blends insights from psychology with economic theory to understand how people actually make choices, which often deviate from the predictions of traditional economic models based on rational decision‐making. Human decision‐making is multifaceted, influenced by emotional, psychological, social, and contextual factors. Yet, so many of the engagement interactions with customers – such as discounts, email campaigns, or customer service calls – are based on a “one‐size‐fits‐all” approach and often face challenges in influencing the nuanced behaviors of consumers and customers.
Human decisions are rarely purely rational; they are interwoven with emotions, cultural norms, past experiences, and even seemingly illogical biases. The static structures that customers interact with – whether online or offline – struggle to keep pace with the dynamism of consumer preferences that evolve with societal trends, technological advancements, and shifts in cultural values. By integrating the power of AI and ML with the deep insights of behavioral economics, organizations can craft more personalized and effective strategies. This fusion enables businesses not only to anticipate and respond to customer needs more accurately but also to create meaningful and engaging experiences that drive loyalty and satisfaction.
Asking the Right Questions: Data, Intuition, and Strategy
Many of the companies and executives whom I have worked with are keen on extracting “actionable insights” from their data to support their intuitions and strategies. However, many have previously collaborated with various consulting firms, which often failed to deliver meaningful insights, leaving them skeptical about new projects. One executive in a Fortune 500 company referred to a collaboration with us as a “last effort to make sense of the data,” and another mentioned that prior experiments and A/B testing had not expedited their quarterly goals. These companies, despite having revenues exceeding $1 billion and vast amounts of data, struggled with leveraging their data effectively.
The common issue is usually the lack of clarity about their assumptions and questions. Although data can easily answer straightforward questions about product popularity or support sophisticated recommendation systems, understanding deeper customer wants and needs remains challenging. Questions about whether customers are comparing costs or educating themselves on options are difficult to answer, complicating strategies for content and product display. This complexity affects sales, product development, and marketing, making it challenging to identify moments for impactful interventions.
Imagine a customer who is shopping for their groceries online from their regular grocer. She spends half an hour selecting 20 items and adding them to her basket, only to leave it all behind. This is curious behavior especially when the prices are competitive, products are available, shipment costs are low or zero, and there are no other clear deterrents in the customers' shopping experience.
Similarly, why does asking to open an account deter customers from completing their purchases? How does concern for the security of their credit card information affect their overall experience? What about factors such as being able to calculate the total order cost easily? How do all these factors affect their trust and sense of transparency affect abandonment rates? These were not mere queries but a narrative that played out repeatedly across many digital aisles.1
The complexity of human interactions and the social and economic networks that we are embedded in begs a deep understanding of the underlying factors that drive behavior and influence decisions. This is why asking the right questions is more important than ever, especially because of the availability of vast amounts of data with high ratios of noise to signal.
Game theory and behavioral economics, the disciplines I am trained in, offer powerful tools and frameworks to understand the complex dynamics of consumer behavior. These disciplines help us uncover hidden assumptions and ask the right questions to identify the true drivers of behavior amidst macroeconomic dynamics and the ever‐changing landscape of consumer preferences.
Game theory, a branch of mathematics and economics, studies strategic interactions between individuals or groups where the outcome for each participant depends on the actions of all. It analyzes how decision‐makers choose their actions to maximize their own benefits while considering the potential choices and reactions of others.
I believe every industry is a customer‐centric industry and it is crucial to understand the motivations and drivers of the observed behaviors, whether they are internal stakeholders or external customers and consumers of products and services. It's not just about understanding isolated instances or scenarios; it's about discerning the intricate tapestry of interconnected use cases, and meticulously mapping and prioritizing the myriad questions that arise from them. Starting with why, through rigorous methodologies and a combination of descriptive, predictive, and causal analytics, we transition to crafting the most fitting models and solutions, guiding toward the optimal action, product, price, or service.
Simplifying the Complexity
At the end of my presentation, an executive, I'll call him John, representing a $1 billion consumer packaged goods (CPG) company approached me. In addition to overseeing a Data Center for Excellence, John also held a pivotal role as a senior executive in sales. He presented a perplexing scenario: During the pandemic, their strategic decision to elevate prices resulted in an unexpected surge in demand. The sales department, although proficient at demand forecasting, was now at a critical juncture. Given the unforeseen increase in demand post the price adjustments, they faced a consequential decision: Should they allocate a significant $100 million toward a new production facility, anticipating continued growth? Would consumer sentiment and purchase patterns remain strong as we approached the end of the pandemic? Or might there be a downturn, necessitating potential staff reductions?
Such dilemmas are not uncommon for professionals entrenched in the CPG sales and marketing sectors. They frequently grapple with substantial decisions, often relying on fragmented and partial data. This is not indicative of a reluctance to adopt a data‐centric approach. Many are, in fact, keenly inclined toward evidence‐based decision‐making. The impediment often stems from data systems that, due to their fragmented structure, cannot seamlessly provide the needed insights. In many large enterprises, even generating a single report can be a prolonged process if it is not already automated. Compounding this challenge is the vast expanse of data, replete with occasionally conflicting insights that executives must meticulously navigate, much like the situation that John was dealing with.
The term data‐driven decision‐making is a buzzword in modern business, but its true meaning often gets lost. What does it really mean to be data‐driven? Which insights are truly valuable for making sound decisions? The first steps in this process always go back to basics: identifying the core problem and asking the right questions.
In the vast sea of data, there are often just a few key points that, when connected, can solve even the most complex issues. It's not about having more data but about finding and understanding the most relevant pieces and how they fit together. This approach, focusing on quality over quantity, is what turns data into actionable insights.
The initial hurdle we confront is the identification, correlation, and combination of the right datasets. The act of curating the right data is intertwined with pinpointing the critical questions we seek answers to. It's akin to a cycle where the inception and conclusion blur into one another: Discerning the right questions invariably illuminates the path to identifying the right data, something that I have come to call “finding the minimum viable data (MVD).” MVD refers to those select data points, often subtle yet profoundly informative, which carry within them the crucial signals necessary for insightful analysis.
This cycle of asking the right questions and curating the right data points is fundamental. When done correctly, it leads to a more streamlined and efficient decision‐making process. The key lies in understanding that more data is not always better. The real power comes from identifying those critical pieces of information that, when analyzed together, provide the most significant insights. This approach not only simplifies the data analysis process but also ensures that decisions are based on the most relevant and impactful information available. By focusing on MVD, businesses can cut through the noise, make more informed decisions, and ultimately drive better outcomes.
Reflecting on the conundrum of positive price elasticity provides a clear example of this principle. To truly understand the impact of rising prices on demand, we must first account for the correct control variables and influencing factors, such as seasonality, macroeconomic conditions, promotional prices, and merchandising. Once these elements are considered, we uncover the reality of negative price elasticity, aligning with established economic theory.
