19,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Fachliteratur
- Sprache: Englisch
Agile is broken. Most Agile transformations struggle. According to an Allied Market Research study, "63% of respondents stated the failure of agile implementation in their organizations." The problems with Agile start at the top of most organizations with executive leadership not getting what agile is or even knowing the difference between success and failure in agile. Agile transformation is a journey, and most of that journey consists of people learning and trying new approaches in their own work. An agile organization can make use of coaches and training to improve their chances of success. But even then, failure remains because many Agile ideas are oversimplifications or interpreted in an extreme way, and many elements essential for success are missing. Coupled with other ideas that have been dogmatically forced on teams, such as "agile team rooms", and "an overall inertia and resistance to change in the Agile community," the Agile movement is ripe for change since its birth twenty years ago. "Agile 2" represents the work of fifteen experienced Agile experts, distilled into Agile 2: The Next Iteration of Agile by seven members of the team. Agile 2 values these pairs of attributes when properly balanced: thoughtfulness and prescription; outcomes and outputs, individuals and teams; business and technical understanding; individual empowerment and good leadership; adaptability and planning. With a new set of Agile principles to take Agile forward over the next 20 years, Agile 2 is applicable beyond software and hardware to all parts of an agile organization including "Agile HR", "Agile Finance", and so on. Like the original "Agile", "Agile 2", is just a set of ideas - powerful ideas. To undertake any endeavor, a single set of ideas is not enough. But a single set of ideas can be a powerful guide.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 739
Veröffentlichungsjahr: 2021
Ähnliche
Table of Contents
Cover
Title Page
Foreword
Preface
1 How Did We Get Here?
A Culture of Extremes
Divided and Branded
Controlled by Dogma
The Introvert vs. Extrovert Problem
Coaches Should Not Assume
Now What?
Notes
2 Specific Problems
Leadership Is Complex, Nuanced, Multifaceted, and Necessary
The Scale Problem
Today's Tech Platform Is As Strategic As the Business Model
Tech Cannot Be an “Order Taker”
Transformation Is a Journey, Not a Rollout
The Individual Matters As Much As the Team
Culture: Individual vs. the Collective
People Don't All Work the Same
Communication Is a Process, Not an Event
The Importance of Focus
Data Is Strategic
Notes
3 Leadership: The Core Issue
Authority Is Sometimes Necessary
The Path-Goal Leadership Model
Collective Governance Does Not Solve the Problem
Dimensions, Modes, Forms, and Directions of Leadership
Soft Forms of Leadership
Applicability and Trade-Offs
Socratic Leadership
Servant Leadership
Theory X, Theory Y, and Mission Command
Knowing When to Intervene
Notes
4 Ingredients That Are Needed
Elements to Keep or Adjust
Elements to Add, Add Back, or Change
Notes
5 Kindsof LeadershipNeeded
Which Leadership Styles Are Appropriate
Leadership at Each Level
Common Types of Product Leadership That Are Needed
High-Risk Products
Research and Innovation Leadership
Operational Leadership
Leadership and Accountability
Any Leader
An Outside Person: A Sketch
An Inside Person: A Sketch
A Person of Action: A Sketch
A Thought Leader
Notes
6 What Effective Collaboration Looks Like
A Collaborative Approach
Respect How Others Work
Team Leads Need to Facilitate Effective Collaboration
Every Interruption Is Costly
Standing Meetings Are Costly
Deep Exchanges Are Needed
The Whole Remote vs. In-Person Thing
How to Make Remote Work
Work
Are Remote Teams a Trend?
Notes
7 It's All About the Product
What to Prioritize
A Product Should Be Self-Measuring
Development System as Product
Notes
8 Product Design and Agile 2
Agile Ignored Design from the Beginning
Technology Teams Need to Be Equal Partners
The Product Owner Silo
The Need for Early and Frequent Feedback
Don't Just Provide Features—Solve Problems
Participatory Design
Single-Track and Dual-Track Approaches
Notes
9 Moving Fast Requires Real-Time Risk Management
The Need for Real-Time Feedback Loops
Creating Real-Time Feedback Loops
Metrics as Feedback
People Need to Understand the Metrics—
Really
Understand Them
Better Information Radiators
People Need to Read, Write,
and
Converse
Validation and Experimentation as Feedback
Flaws in the Pipeline Model
Feedback: Learn from Product Usage
Balance Design and Experimentation
Responding in Real Time
Notes
10 A Transformation Is a Journey
Agile Is Not a Process Change
What a Learning Journey Looks Like
Organizational Inertia Is Immense
You Do Not Need to Build Anything
Notes
11 DevOps andAgile 2
What Is DevOps? And Why Does It Matter?
Common DevOps Techniques
Data
Notes
12 Agile 2 at Scale
Issues That Arise at Scale
What Is the Strategy?
Strategy and Capability Alignment
Portfolio and Capability Intersection
Need for Hierarchy
Initiative Structure and Leadership
Coordination at Scale
R&D Insertion
Multiple Stakeholders
Knowledge Gap
Reflection on FamilyLab
Notes
13 System Engineering and Agile 2
How Hardware and Software Differ (or Not)
Multitier Products and Systems
Our Case Studies
Case Study: SpaceX
Case Study: A Major Machinery Manufacturer
Notes
14 Agile 2 in Service Domains
Define a Target Culture
Process Varies with Circumstances and Time
The Dysfunction of Staffing Functions
Balance Short- and Long-Term Views of People
Design an Effective Work Environment
Assess Performance Immediately
Notes
15 Conclusion
A Model for Behavioral Change
No More Tribalism
Agile Cannot Be Simplified
Agile Is Timeless
Notes
Index
Copyright
Dedication
About the Authors
Acknowledgments
End User License Agreement
List of Tables
Chapter 4
Table 4.1 Building Construction Compared to Software Development
List of Illustrations
Chapter 1
Figure 1.1: Blinders to help people focus in “team rooms”
Chapter 9
Figure 9.1: Continuous builds being delivered, some potentially usable, some...
Figure 9.2: Simple pipeline
Figure 9.3: Simple pipeline—each step balanced
Figure 9.4: Two pipelines—cadence-based
Figure 9.5: Two pipelines—on demand (no waiting)
Chapter 12
Figure 12.1: Inconsistency of Spotify features across platforms
Chapter 13
Figure 13.1: Team interaction modes
Guide
Cover Page
Table of Contents
Begin Reading
Pages
iii
xv
xvi
xvii
xix
xx
xxi
xxii
xxiii
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
iv
v
vi
vii
viii
ix
x
xi
406
Agile 2
The Next Iteration of Agile
Cliff Berg
Kurt Cagle
Lisa Cooney
Philippa Fewell
Adrian Lander
Raj Nagappan
Murray Robinson
Foreword
You can recognize a great book by its ability to make obvious what is wrong with existing worldviews and to add new insights or nuances to change or improve this worldview. I believe Agile 2 meets these criteria easily. It is an easy read not only for those new to the subject of agility, but also for die-hard professionals who are looking for something “beyond” the basic Agile concepts that are at times dogmatic and often being misused in practice.
In this era of digital disruption and an ever-growing world full of volatility, uncertainty, complexity, and ambiguity, it becomes increasingly important for organizations to become more agile. The Agile Manifesto, originally intended in 2001 to disrupt traditional, not-too-effective software development practices, has inspired many organizations over the past decades to change their ways of working, affecting both work cultures and structures. Its popularity was boosted by a growing workforce consisting of millennials and Generation-Z professionals who demand more autonomy, ownership, and the opportunity to make meaningful impact. Over the past decade, the “Agile movement” has gained increasing momentum and also moved beyond the realm of IT.
A side effect of its success and growth was that all kinds of Agile frameworks, doctrines, and certifications popped up to standardize and monetize the discipline. The original Agile values and principles, being high-level on purpose, gave ample room for various interpretations of the core paradigms. During my career I have worked with many different Agile coaches and consultants, and I was always surprised by how much discussion and fanatic debates arose among them with regard to how to live certain Agile values or implement specific practices. This tribalism led to confusion among non-Agilists, and this hampers Agile transformations significantly. I thus see a clear need for a comprehensive Agile idea set that is both pragmatic and nuanced by nature. Enter Agile 2.
I was happy to find out quickly that the authors do not claim to have written yet another Agile doctrine or “Bible.” Instead, they have written a pragmatic companion guide that will be useful for managers and specialists alike. It is packed with hands-on tactics and practices that can help leaders and specialists in organizations to grow to a next level of agility, while preventing cargo-cult behavior or avant le lettre implementations that often do more harm than good.
One of my key drivers for cofounding the DevOps Agile Skills Association (DASA) in 2016 was building a comprehensive view on how to create high-performance IT organizations. The popularity of DASA stems largely from the six DevOps and Agile principles that advocate continuous improvement, customer centricity, autonomous multidisciplinary teams, and product thinking. Following these principles often results in a digital and organizational transformation that typically goes far beyond choosing a standard Agile framework or adding some basic Agile rituals to the mix. To transform successfully to high-performance, organizations need a more mature take on and guidance on what it means to really “be Agile” at scale. Providing this guidance is one of this book's core differentiating features.
Over the past decade I learned firsthand as a consultant, trainer, and senior leader the importance of building the right type of leadership in the organization and creating a culture of continuous learning, experimentation, and innovation. What I like about Agile 2 is that both the importance of leadership and learning are advocated strongly. It provides many tangible ideas to reimagine an organization's leadership culture. I wish that I had this book on my nightstand five years ago. It would have helped me greatly in understanding why certain things happened—or did not happen—during the organizational transformations I was leading.
I like the fact that the authors do not intend to reinvent the wheel, but are keen on building on what is already working. Some of the key Agile values and principles are powerful to this day, but application in practice often needs some additional clarity and lots of examples. The authors nicely provide nuance to how to interpret Agile principles and values while referring to many interesting, and more recent, bodies of work. The authors hit the nail with addressing key topics that are haunting many organizations, leaders, and teams, such as how to collaborate, communicate, value both experts and generalists, and commit team capacity. They rightfully argue that how to adopt certain principles or how to interpret certain values depends on your organization's needs and its current level of maturity. Using this book as your Agile guide, you can aim and navigate your transformation in a more tailor-made way, resulting in more business value. I expect this book to be found on many nightstands in the coming years.
Dr. Rik Farenhorst
Senior IT Exec | Trainer | Coach | Speaker | Writer on Creating High-Performance Digital Organizations | Co-founder of DevOps Agile Skills Association (DASA)
Utrecht, The Netherlands
December 2020
Preface
A few people who have become aware of Agile 2 have dismissed it as “more of that Agile stuff,” not realizing that Agile 2 is a departure from the original Agile in attitude, approach, and substance. One of those individuals—a chemical engineer—said that he had discussed Agile at length with an Agile advocate, but still concluded that Agile is not for him. Another—an experienced systems engineer who has testified before Congress regarding aircraft and spacecraft systems reliability—also believes that Agile methods do not provide a robust process for trustworthy systems.
We view ourselves as Agilists, and yet we find widespread doubt about the efficacy and usefulness of Agile in many quarters. One of these is among engineers. These people are not ignorant. They know their job extremely well, yet Agile, as described to them, or as they have experienced it, has not resonated or has not answered critical questions.
The Agile movement also uprooted the product design community to some degree (which we will document in this book), although this is an area in which the Agile community has realized the issue and some are trying to rectify it.
Agile authors largely ignored the role of data: something that is so immensely important, that it is akin to speaking about mountains but missing a vast canyon immediately beside you.
The Agile community also sidestepped the issue of leadership — something that the DevOps community has tried to address. Leadership is so important for any endeavor, that to omit it is, frankly, quite equivocal.
Agile has not resonated among the growing DevOps community. Even though DevOps ideas were developed by people who strongly identified as Agilists, the Agile community at large has remained mostly ignorant of DevOps, which had the effect that DevOps became its own movement. As a result, most Agile coaches today know little about DevOps, and we find that DevOps practitioners often view Agile as superfluous.
You might think that mainstream programmers accept Agile, since they are the ones who use it most directly, but in actuality, there is a lot of doubt about Agile within programming communities in general. That is the biggest irony of all: that Agile, which was created for programmers, has in effect been taken away from them, and no longer serves them.
Agile is mostly accepted within Agile communities—comprised of Agile coaches, and managers who have been persuaded of the benefits of Agile. Programmers tend to have mixed feelings about Agile. (We will support that assertion in this book.)
Was Agile described poorly? Is Agile missing things? Did it get some things wrong? Does Agile truly not apply to the needs of the work of any of these people? Since Agile ideas can be applied to most things (in our opinion), we believe that the last explanation is not likely to be the true one.
What we have observed ourselves is that too often, Agilists explain and advocate Agile ideas and methods before asking enough questions. Some of us have seen Agile coaches fired for coming into a setting and insisting on particular practices before actually understanding how the work in that setting is done—a hypocrisy given that Agile coaches so often explain that Agile transformation is a learning journey.
To understand how to apply Agile ideas, one must first understand that domain, how the work is currently done, and why it is done that way. No Agile practice is universal. One size does not fit all, so prescribing before understanding is potentially destructive.
Indeed, the “dogma” of the Agile community helped to launch it, but it has also been its chief failing. Early proponents of Agile insisted that the Agile movement needed to be disruptive—a “call to arms”—and so dogma was called for; but dogmatic insistence also alienates and causes dysfunction when it is not the best advice for the situation.
Agile 2 is not dogmatic. It is not designed to stir up emotion. It is not a call to arms or an attempt to disrupt what we have. As such, it does not try to be disruptive. It does not replace Agile or replace DevOps or replace anything. Agile 2 pivots Agile in some important ways and attempts to fine-tune it. Agile 2 also adds many extremely crucial ideas that have been ignored by much of the Agile community, even though successful Agile practitioners often use those very ideas, and other communities of thought embrace those ideas.
Agile 2 reinforces some DevOps ideas, and some Lean ideas, but Agile 2 does not attempt to duplicate or replace those, and so those sets of ideas are still important in their own right. Agile 2 does not attempt to subsume any existing community of thought. Agile 2 also does not claim to cover all aspects of these topics. Agile 2 claims to only be a set of useful ideas for how to achieve agility in human endeavors and encourages people to include other ideas and fields of thought as well.
Agile 2 is more verbose than the Agile Manifesto. The reason is that we feel that one of the weaknesses of the manifesto was that it over-simplified complex issues. A simple value maxim cannot describe important trade-offs, and one or two principles cannot address nuanced issues such as what good leadership looks like and which styles of leadership apply best in a given situation. So Agile 2 gives these important topics the space they deserve, from an agility perspective.
One important way that Agile 2 departs from the Agile Manifesto is that Agile 2 provides the foundation of thought from which it was derived. Rather than make bold statements without substantiating them, Agile 2 provides the “problems” and “insights” that arose in discussions about Agile, in the course of the Agile 2 team's retrospective about the state of Agile. It was these problems and insights that led to the Agile 2 principles.
An Agile 2 principle is not intended to be an absolute. This is because there can be no absolutes when it comes to human behavior. An Agile 2 principle is a proposed rule of thumb: true most of the time, but perhaps not in some circumstances. That is why the underlying assumptions and thoughts—the problems and insights—are important for understanding the intention of each principle, meaning what problem it is trying to solve and how.
Someone posted a comment online about Agile 2, saying that if the original Agile Manifesto authors were not involved in the Agile 2 effort, he would not look at it. We believe that all great ideas build upon what has come before, and that even “original” ideas have deep roots. The term Agile had been used prior to the creation of the Agile Manifesto, and “agile” methods had been used and circulated for years prior to that. Not only do many Agile methods date back decades, but core ideas in Agile 2 such as Socratic leadership date back millennia.
There is also the matter of dysfunction within the Agile community, which we will discuss at length in the book. The dogma that is found in some quarters is one form of dysfunction; another is the separation of the community into tribes, for the various frameworks. We will explain why this has been a problem and how it has “frozen” Agile thinking and stifled its evolution.
Many of the thought leaders in the Agile community have a lot invested in current paradigms and practices, and so change is not in their best interest. For these reasons, we felt that we could not rely on the community to fix these problems. The problems come from the community—not from the whole community, but from some of the most established and entrenched parts of it.
Why bother then? Why deal with this? It's because Agile is extremely important. DevOps cannot replace Agile. While Agile has become mostly about the human side of building things, DevOps has become mostly a collection of technical practices. That is the reality on the ground. But there is more to building things than the technical side: one needs both the human side and the technical side.
We therefore realized that addressing Agile's gaps is really important, and that to do it, we needed a diverse team with a wide range of skills, composed of people who are not deeply invested in current paradigms or frameworks. We needed original thinkers. Those criteria led to the Agile 2 team of 15 members, which you can find on the Agile 2 website.
Agile 2 is an attempt to make a solid course correction to Agile, but in an open, additive, inclusive, and nondogmatic or emotional way. We welcome ideas that can supplement Agile 2 and feedback on its principles, in the spirit of inclusiveness and advancing everyone's understanding of these complex issues.
Agile 2 broadens Agile's focus beyond software. The reality is that Agile ideas have been applied for many things besides software, and so the Agile 2 team felt that it made no sense to define Agile 2 only for software.
The reader will notice that many Agile 2 principles are stated in the margin, but not all of them. This book is not a textbook about Agile 2 that covers every aspect of its principles. The purpose of this book is to introduce Agile 2, explain why we need it, and give an overview. You can find more information about Agile 2 on the website: https://agile2.net, which is published under a Creative Commons Attribution license, “CC BY 4.0.”
This book attempts to make the many topics of Agile 2 concrete. We give guidance on how to apply the principles and provide examples. However, we refrain from providing specific steps or templates to follow: we do not want to repeat the mistake of current Agile frameworks in that regard.
Except for our examples, we do not define practices to implement. Practices are important, but that is for another book and for others to propose. This book lays out a conceptual foundation, while using concrete situations as examples, but not for prescription.
A note about the use of the words “agile” and “Agile”: This book uses the word “Agile” when referring to the ideas embraced by the “Agile community,” which is comprised of Agile coaches and others who view the agilemanifesto.org document as a guiding source of insight; or in the context of so-called “Agile frameworks” which claim to define practices that are consistent with the philosophies of the Agile community. We use the word “agile” when we intend to convey the generic quality of agility. Agile with a capital “A” and agile with a small “a” are two different words.
The name “Agile 2” is not intended to be a version number, as in “2.0,” “2.1,” etc. Rather, it is the name of a reborn Agile. We feel that the principles of agility are timeless, so we do not expect an Agile 3, and so on. Rather, we see Agile 2 as an attempt to reimagine Agile—not from scratch, but by taking Agile ideas and pivoting. We hope that Agile 2 hits closer to the mark!
1How Did We Get Here?
At a developer conference in 2015, Dave Thomas, one of the authors of the Agile Manifesto, gave a talk titled “Agile Is Dead.”1 In a 2018 blog post, Ron Jeffries, another Agile Manifesto author, wrote, “Developers should abandon Agile.”2 In a 2019 article in Forbes titled “The End of Agile,” tech author Kurt Cagle wrote, “[Agile] had become a religion.”3 A post about the article4 in the programmer forum Slashdot received more than 200 comments from software developers, asserting things like “Agile does not always scale well” and “The definitions of ‘agile’ allow for cargo cult implementations.”
Agile has been a subject of ridicule since its beginning. In the early days, there were many people who did not understand Agile and spoke from ignorance; what has changed is that today the criticism often comes from people who do understand Agile methods and have decided that those methods are problematic.
Is Agile actually dead? The statistics say no,5 yet something is clearly wrong. Agile—which was sold as the solution for software development's ills—has severe problems. What are those problems, how did they happen, and what can be done about them? And is Agile worth saving?
Most of the discussion in this chapter will be about software. That is because Agile began in the software domain. In later chapters, we will broaden the discussion to product development in general, and to other kinds of human endeavor, since many Agile ideas apply to essentially any group effort.
A Culture of Extremes
In 1999 a new book called Extreme Programming Explained by Kent Beck sent shock waves through the IT industry. Agile ideas had been circulating and in use prior to this, but Beck's book somehow pierced corporate IT consciousness. It arguably launched the Agile movement, even though the movement was not called “Agile” yet.
The movement's core thesis was that methodical, phase-based projects were too slow and too ineffective for building software—challenging the approach then used by most large organizations and pretty much every government agency.
The book did not launch Extreme Programming, aka XP, which was first defined in 1996,6 but it was the book that popularized it. Talk about XP could be heard in the halls of every IT shop. It was controversial, but its values strongly resonated: Small teams, working code (rather than documents) as the only real proof of progress, frequent discussions between the customer and the programmers. Out with big, up-front requirements documents that were always incomplete, inconsistent, and incomprehensible; out with big, up-front designs that were usually wrong. Recurring and incremental customer approval instead of contracts that locked everyone in to unvalidated assumptions.
Many of the methods of XP were not new, but they had been outlier methods, and XP put them under a single umbrella. The book strongly asserted that these methods work and are a superior alternative to traditional methods.
It is not that there were no other proposals for how to reshape software development. So-called lightweight methods had been around for a while. Extreme Programming was new in that it threw a grenade into much current thinking by being so radically different and proposing methods that were so extreme—methods such as pair programming (which had been described as early as 1953)7 and Test-Driven Development (which also had some history prior to XP), which turned many assumptions about programming on their head.
Thus, the movement began as a rejection of the predominant existing paradigms. People knew something was wrong with software development as it was being done. Extreme Programming provided an oppositional alternative. It was not so much that people thought XP was great, but they were sure that current practices were not great. XP received a lot of attention and was a radically different approach. Perhaps the attention was not because XP was so much better or radical, as there had been other ideas circulating such as Rapid Application Development, but perhaps XP got attention mostly because the Internet provided a new medium that made rapid awareness possible.
Then in 2001 a group of IT professionals—all men by the way, with most from the United States and a few from Europe—got together over a weekend and hammered out a set of four “values,” which they believed should be the foundation of a new approach to building software. Kent Beck was among them. You can find these four values at AgileManifesto.org . It was largely a rejection of many approaches that had become commonplace, such as detailed plans, passing information by documents, and big all-at-once deliveries.
In the weeks that followed, some of them continued the discussion by email and added 12 principles, which you can also find at the same website.
They called all this the Manifesto for Agile Software Development, and it came to be known colloquially as the Agile Manifesto or just Agile. This “manifesto” took the popular culture baton from XP and other iterative approaches and launched the Agile movement for real.
Extreme Programming had set the tone for what would become the Agile movement, and the tone was to be extreme. In those days, extreme was popular. We had extreme rock climbing, extreme skateboarding, extreme pogo, extreme skiing, and extreme pretty much anything. Extreme was in. People were so tired of the ordinary; everything new had to be extreme. It was a new millennium for crying out loud: everything needed a reset!
And so “extreme” was a necessary aspect of anything new and interesting at that moment in time in the late 1990s—the end of the 20th century.
Since Agile was a rejection of what had become established software development methods, it was inherently a disruptive movement, and in the ethos of the time, it had to be extreme. And so it was that every Agile method that came to be proposed—these are called practices—were of necessity extreme. Otherwise, they were not seen to be consistent with the spirit of being entirely new and disruptive.
It was not the Agile Manifesto that set things in that direction. The Agile Manifesto was clearly about balance and moderation. It makes no absolute statements: every value is couched as a trade-off. For example, the first value reads, “[We have come to value] individuals and interactions over processes and tools.”
It does not say, “Forget process and tools—only pay attention to individuals and interactions.” Instead, it says, consider both, but pay special attention to individuals and interactions.
In other words, the Agile Manifesto advocated judgment and consideration of context. In that sense, it is a sophisticated document and cannot be used well by people who do not have the experience needed to apply judgment.
But the tone had already been set by XP: extreme practices received the most attention and applause, because XP practices were all extreme. For example, XP's recommendation of pair programming, in which two people sit together and write code together, sharing a keyboard, was considered by many programmers to be extreme. Or everyone sitting side by side in a single room, with all walls removed and no privacy—that was pretty extreme, as it had been assumed that people needed privacy to focus, and the big programmer complaint of the 1990s, depicted in so many Dilbert cartoons, was that programmers were no longer being given offices and instead were being sat in cubicles that did not afford enough quiet or privacy. And now here comes XP and says, in effect, You got it all wrong; you need to sit next to each other. That was an extreme swing of the pendulum.
The Scrum framework, which dates in various forms to the 1980s but became reformulated in 1993 and then popularized through its certification regime during the 2000s,8 added more ideas that are arguably extreme. For example, Scrum views everyone on a team as an equal player—no one is acknowledged as having more standing than anyone else (“Scrum recognizes no titles for Development Team members”9), regardless of their experience. That was pretty extreme, since before Agile, programmers in most (not all) organizations had professional levels, such as programmer, senior programmer, architect, etc.
During the first decade of the Agile movement it seemed that new suggested practices were in a competition to be more extreme than the others. We saw the introduction of mob programming, in which rather than two programmers working together as in pair programming, the entire team works together—literally—everyone calling out their thoughts in a single room and sharing a single keyboard.10 Then in 2015 the Agile team room was extended by Facebook to an extreme level when it created its 430,000-square-foot open team room.11
We also saw the growth of conference formats pushing popular Agile practices to the extreme—for example, the Lean Coffee format in which there is no agenda or the “unconference” and Open Space formats in which people vote on topics and join ad hoc conversations. These contrast strongly with a traditional conference or discussion group format, which generally has an agenda and scheduled talks, with informal sessions afterward.
But do extremes work?
Certainly they work for something. There is always some use for any tool. The question is, are the extreme practices advocated by many among the Agile community actually the most effective method for a wide range of situations that commonly occur in organizations? Another question is, do these practices favor certain ways of working at the expense of others so that certain people benefit but others are at a disadvantage? Or, should a thoughtful approach be used, with moderate approaches being the norm and extreme approaches used sparingly and when a particular form of activity is desired for a specific reason?
Since the Agile movement began through advocacy of extremes and was inherently a disruptive movement, it became evangelistic, and dogmatic elements arose. As a result, if one did not embrace the trending favored set of Agile practices, one was at risk of being labeled an “Agile doubter.” That label was brandished readily by many Agile coaches. And so extremes came to be not just something to consider, but the way—and the only way. Agile was now something to accept on faith; as Kurt Cagle had written, it had become a religion.
Divided and Branded
Whenever something new and useful is created, people and organizations jump in to claim it and use it for their own purposes. Any change creates huge opportunities. For example, when Howard Head came out with the oversized Prince tennis racquet in the early 1970s, other manufacturers followed Head's lead and came out with racquets that departed from the standard size. One suddenly saw racquets on the market with very large nets and also ones that were only slightly larger than what was then the standard size.
The ones that were slightly larger became the most popular and came to be seen as the new standard size. These were called “midsize” racquets, although today we just call them racquets.12
The inevitable result was that everyone who owned a tennis racquet had to go out and buy a new one; otherwise, they were not adhering to the new “standard.” The sports equipment industry experienced a windfall in sales.
The rise of Agile did the same thing. There were new books, new websites, new consulting practices, and new frameworks that purported to be Agile. Agile quickly became a commodity to sell. It began with an Agile certification industry. Ken Schwaber introduced a two-day certification course for his Scrum framework: the Certified Scrum Master, aka CSM. By sitting in a training room for two days and not even taking a test (they do provide a simple test now), one could walk away with a certificate claiming to be a “master.”
Essentially the material that could be contained in a small pamphlet was the basis of what Human Resources staff and many hiring managers erroneously interpreted as a “master-level” certification.
Since Schwaber and his partner, Jeff Sutherland, claimed that Scrum was an Agile framework, it was something they could sell under the rising banner of Agile. Organizations that preferred to hire people with certifications made the CSM a requirement. People who had master's degrees from universities were dismayed at the naively perceived equivalence of a master's degree and a Certified Scrum Master certification. Highly qualified people were screened from job applications because they did not have the two-day CSM certification.
Industry groups sprang up: the Agile Alliance, the Scrum Alliance, Scrum.org , ICAgile, SAFe, LeSS, Kanban, and many others. The large consulting companies had a hard time learning about Agile, because Agile's central message of being lean and efficient and not having big contracted “phases” was antithetical to their model. Eventually they figured out how to incorporate it into their offerings, and today they all have substantial Agile practices, claiming to be the experts in Agile.
Thus, there is a lot of money today in the Agile industry, and the Agile community is arguably driven by moneyed interests, with a continuing tide of people seeking certifications in the various frameworks and becoming indoctrinated into them. The phrase Agile industrial complex, which might have been coined by Martin Fowler (one of the authors of the Agile Manifesto), has come to be used to refer to the industry as a commentary on the degree to which it is driven by financial interests rather than by ideas and efficacy.
Controlled by Dogma
Today Agile is big business, and it is highly competitive.
Large consulting companies have traditionally used their partners to fly around and build relationships with their clients' executives, convincing the executives that the partners and consulting firms have strategic insight. That was the case before COVID-19 and will probably resume being the case after COVID-19. Through those relationships, the partners are able to place large numbers of Agile-certified staff on-site, generating a lot of revenue.
Placing staff is their goal—as many as possible. These staff members do not usually have the industry experience that the partners have, but they have a certificate, perhaps a Certified Scrum Master certificate, perhaps a SAFe certificate, or perhaps others. In other words, they sat in a classroom for a few days or weeks, and they probably participated in at least one project that was said to be Agile. Most are not what one would normally expect from a consultant who is advising teams and programs: decades of practical experience at multiple levels of responsibility, great acumen, demonstrated industry thought leadership, and a history of P&L accountability.
Certainly there are many people in large consulting companies who are very qualified, but we have seen many who are not. The point is that one should not treat a large consulting company as if it is inherently more trustworthy than any other with regard to Agile expertise. Have their people achieved business results? The partners will tell you they have and will provide proof, but you know how data can be misused.
Many of the various industry groups do not like each other either. One of the largest Scrum training organizations—one that claims to speak for the Scrum community at large—actually writes into its contract with its trainers that one cannot train for a “competing” training framework such as the Scaled Agile Framework. Thus, the organization does not want its customers to have a choice or to be able to consider other ideas. Limiting what one’s trainers know does not seem to us like a good way to serve one’s customers.
Meanwhile, the organization preaches Agile's value of “customer collaboration over contract negotiation” and professes to being open to all ideas and to being cooperative and collaborative—core Agile attributes. You should not miss the hypocrisy in this.
An Agile training organization's armies are its certified coaches. When the CSM certification was introduced, it had an unintended effect (at least, there is no evidence that it was intentional). Large numbers of people who had no software development experience took the training and became certified. Almost overnight they had a new career: their CSM certification could get them a job as a Scrum Master or an Agile coach.
We then saw the rise of these two new professions (Scrum Master and Agile coach), and they quickly came to be dominated by nontechnical people from every manner of prior profession. Since their background was nontechnical, they emphasized the human side of software teams: team trust, team happiness, and so on, as well as the set of Scrum practices: the sprint planning, the daily scrum (aka daily standup), the sprint review, and the retrospective. The technical focus that XP had was almost entirely lost, and by 2011, 52% of Agile teams reported using Scrum (which defines no technical practices), compared to only 2% using the highly technical XP.13
This is a big problem. In a talk at Agile Australia in 2018, Martin Fowler, one of the authors of the Agile Manifesto, asked the audience, “How many people here are software developers?” Then he said, “A smattering, but actually very much a minority … and that's actually quite tragic.”14
Agile coaches and Scrum Masters focused on the nontechnical practices as if they were the be-all and end-all of software development, ignoring critical things such as test coverage, code branching, integration testing, issue management, and all the critical things that every programmer knows are essential. The retrospective, in which a team is supposed to talk about how to improve their work, was facilitated by the Scrum Master, who was usually nontechnical, and so the discussion tended to steer toward the Scrum practices, because team members did not want to bring up issues that the Scrum Master would not understand. Teams then went back to their desks, anxious to get back to work after all of the Scrum-related ceremonies (what the Scrum practices are called), and so important technical issues went undiscussed.
The Agile conversation was essentially taken away from software developers—the people for whom Agile was created. After all, the Manifesto begins with (italics added), “We are uncovering better ways of developing software … .”
Agile thought leadership was increasingly controlled by those who wanted to advance an increasingly extreme behavioral agenda. Programmers speak up from time to time, but with trepidation. For example, a poster on Reddit wrote this:
“I hate Scrum. There. I said it. Who else is joining me? Scum seems to take away all the joy of being an engineer. working on tasks decided by someone else, under a cadence that never stops. counting story points and ‘velocity’. ‘control’ and priority set by the business - chop/change tasks. lack of career growth—snr/jnr engineers working on similar tasks.”15
The Scrum Master and Agile coach roles became so entrenched that organizations now assume that they need those roles. No one questions it. More important, no one questions the skills and knowledge that are needed for those roles to actually be effective.
The career of a Scrum Master or Agile coach relies on the perceived value that they provide. Since the experience of most was not in the technical realm, the natural evolution was to inflate the value of the nontechnical dimension of things. We saw a rise in Agile dogma, in which Agile or Scrum advocates—usually Agile coaches or Scrum Masters—would deride anyone who challenged Scrum or Agile practices. We have mentioned the term Agile doubter. These dogmatic people were such a problem that in a 10-year retrospective on the state of Agile, the British Computer Society referred to Scrumdamentalism—a play on the words Scrum and fundamentalism—as one of the main problems with the state of Agile.16
The Agile community suffered from groupthink: the community mostly spoke to itself. One of our editors described the community as insular. During the first decade of Agile, Agilists mostly read books that said “Agile” on the cover, but important ideas from other communities, such as those that study organization change or organizational psychology, did not permeate the Agile community. In the words of Norwegian Wood's author Haruki Murakami, “If you only read the books that everyone else is reading, you can only think what everyone else is thinking.”
The result was that thinking in the Agile community became highly constrained: the community would only tolerate ideas that either modified existing Agile practices to be more extreme or added new nontechnical practices that were also extreme. Anything else was either ignored or dismissed as “not really Agile.” So for the entire decade of the 2000s, Agile was confined largely to nontechnical thinking—an irony since Agile was created for software development.
Meanwhile, companies such as Google, Amazon, and Netflix had real problems to solve. They needed to be able to deploy to hundreds of millions of users many times a day, with confidence, and allow teams to make rapid changes and redeploy without delay. They needed agility in their software development and release processes.
So, they invented techniques to help them to achieve these things: techniques such as on-demand test environment creation, test-first integration testing (aka ATDD), containerization, container cluster management, and many other things. Jez Humble was the first who we know to have assembled these ideas into a coherent whole and publish a book about them, which he called Continuous Delivery, and the methods he described became known as continuous delivery, often abbreviated as CD. Gene Kim later wrote about CD ideas in his book The Phoenix Project.
Humble was an advocate of Agile methods and viewed himself as an Agilist, and he spoke about his work at Agile conferences. His talks were well attended, but few in the Agile community at large paid attention: what Humble was advocating was technical, so it was largely ignored by the large and growing cohort of nontechnical Agile coaches, particularly those who had built a career around Scrum. The established spokespeople for Agile—the popular Agile authors of the day—also largely ignored what Humble was talking about, preferring to pile onto the process topics that had been trending, and a few kept tweaking the now dated practices of XP. As a result, CD methods and related ideas became their own movement, which we know today as DevOps—a term that arose shortly before Humble's book was published.
The Agile community saw the rise of DevOps in the early 2010s and became alarmed. Agile was under threat from something new, so they claimed it as their own. Those who still advocated for XP claimed that DevOps was just XP in a more evolved form (not true). Agilists pointed to Humble's Agile origins but did not acknowledge that the majority of Agile coaches paid little attention to CD until DevOps rose to widespread awareness.
To be fair, there are many technical Agile coaches, even though they are a small minority. Those people indeed saw CD as continuation of the march of Agile ideas that began with XP and before. They were a fractional cohort, though.
Today if you ask an Agilist about DevOps, they usually think that DevOps is an evolution or outgrowth of Agile. That is not how it was, but it no longer matters. Agile is a set of ideas. DevOps is a set of ideas. Both sets of ideas, and many others, are extremely valuable in the context of product development.
The Introvert vs. Extrovert Problem
Earlier we posed the question, do these practices favor certain ways of working at the expense of others so that certain people benefit but others are at a disadvantage?
The group that authored the four values of the Agile Manifesto was pretty homogeneous. As we said, they were all English speaking, most were from the United States with a few from England and one from the Netherlands, all were men, and all were very experienced. They did not represent a “typical” team of programmers. It is reasonable to expect, therefore, that they had some collective biases that differ from how a typical team of programmers would feel and think.
Since most were experienced, one might assume that when they wrote that they value “individuals and interactions over processes and tools,” they were thinking of themselves as capable individuals—individuals who are highly experienced and who have refined judgment about IT methods. Such individuals arguably need less oversight—less process.
Within the principles that some of the group came up with, one of them reads, “The best architectures, requirements, and designs emerge from self-organizing teams.”
That one principle has resulted in a strong preference within the Agile community for “self-organization,” that is, a team in which there is no designated leader. The authors of the Agile Manifesto were able to come up with four values in the course of a weekend, so they clearly were able to self-organize enough to accomplish that. Would they have been able to continue to be organized over a period of months to create a complex software product?
No one can say. The principle regarding self-organizing teams has been one of the most contentious statements of the Agile Manifesto. An article by one of us in LinkedIn about self-organizing teams received 45,000 reads in the first two weeks—two orders of magnitude higher than the normal read rate for that author.
What happens if you put a group of people together and tell them to “self-organize” to achieve a goal? Well, it depends.
You will read that phrase, “it depends,” a lot in this book. We believe it is central to all of the issues pertaining to how groups of people should work together. In the words of Malcolm Gladwell, the author of Blink, the only answer that is always right is “it depends,” and that is definitely true when dealing with groups of people.
Some of the factors affecting how well a group self-organizes are:
Personalities
—some personalities mix well, others less so.
Level of knowledge and experience
—do they know the job well enough to get it done without supervision?
Urgency
—how important is their goal? Do their lives depend on it? Or at the other extreme, is it something that someone else wants—something the team sees as inconsequential to them—and they are mere mercenaries?
Preparation
—have they been trained to work in a certain way so that everyone has predefined roles?
Proven history
—have they worked together before and shown that they can?
There are certainly other factors as well.
Some people work well in a group; others have trouble in a group and prefer to have their own task. The Agile community tends to dismiss the value of these “loners” or “lone wolves.” Yet lone wolves have created some of the most impactful software. Linux is an example. It was created by Linus Torvalds, who is a self-proclaimed loner and introvert. Today he still oversees the evolution of Linux, but he does so in a room by himself, communicating with the thousands of Linux kernel developers involved entirely by email. Linux powers most of today's computers, from smartphones to most of the servers in the cloud. If you use Amazon, you use Linux. If you use an Android phone, you use Linux.
Introverts tend to shy away from groups. Groups are taxing for them. In contrast, extroverts seek out groups—groups energize them. It follows then that in a team in which the prescribed form of communication is face-to-face and verbal, introverts would feel a lot more taxed than extroverts, who would actually find the frequent face-to-face discussions energizing.
One of the principles in the Agile Manifesto reads, “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”
This seems to clearly favor extroverts. Extroverts will enjoy resolving every issue by turning around and talking to others. They would enjoy convening an ad hoc meeting to discuss the issue. According to Noa Herz, a neuroscientist and neuropsychologist, “Group meetings, in which each participant contributes thoughts in a disorganised, dominance-based manner, can put introverts at a disadvantage.”17
An Agile coach or Scrum Master is supposed to facilitate team meetings, but doing so can be challenging. If a few members are talking rapidly, a facilitator might feel that the members are “on a roll” and be hesitant to interfere.
An introvert—by definition—will find that situation emotionally taxing. Even if asked their opinion, they are not likely to keep the floor long enough to actually convince anyone. It is therefore reasonable to assume that introverts might prefer to first communicate in written form and meet in person only if it is necessary.
An interesting fact about programmers as a group is that they think they are extroverts.18 Yet some psychological studies indicate that programmers actually tend to be introverts.19 It is a complex issue, though, because other studies produce conflicting results on the question.
Regardless, there is still a problem, because while Agile appears to favor extroverts, the people at whom Agile methods are mostly targeted—software developers—have a high proportion of introverts, as a group, even though the actual percentage is unclear. Even if, say, 60% of programmers are extroverts—something that would be a counterintuitive career choice—forcing extrovert-favoring methods on the other 40% would be unfair and counterproductive.
It is also a problem because by sidelining introverts, teams suffer. According to Herz, “Entrepreneurial teams perform better when leadership is shared between individuals, but only if they have diverse personality traits. Moreover, teams dominated by extraverted members actually perform better under introverted leaders,20 possibly due to their greater responsiveness to their employees' ideas.”
How did the Agile Manifesto come to be written to favor extroverts? Were the authors of the Agile Manifesto extroverts? It is hard to say, but the manifesto's stated preference for face-to-face communication aligns with what one of the authors had been writing shortly before their meeting. Alistair Cockburn had written extensively about how, in his opinion, face-to-face communication is the “most effective” form of communication.21
So it is possible that one member of the group influenced the others—a member who seems to fit the profile of an extrovert, based on his high level of involvement in Agile community group initiatives.
Is it really true? Is face-to-face communication always best?
It depends. (There it is again.) We will dig into that question a great deal in Chapter 6, when we talk about how collaboration occurs for simple and complex issues.
What about the people who help Agile teams to define how they work? That is, Agile coaches and Scrum Masters? Do they tend to be introverts or extroverts?
We are not aware of a study on that question, but consider that in seeking such a job, one is looking for a role in which one is coordinating groups of people and encouraging face-to-face communication. It stands to reason that the job, as defined by Scrum and the Agile community, would attract people who like working with groups of people—in other words, extroverts. And it has been our experience that Agile coaches do, by and large, seem to be “people people,” although they are as varied as the people in any profession, and no two are the same.
If it is true—if Agile coaches and Scrum Masters lean slightly extroverted—then we have a slightly extrovert-leaning population advising a somewhat introvert-leaning population on how to work. What could go wrong?
Ignoring whether an Agile coach or Scrum Master is an introvert or extrovert or something in between, and following default Agile practices that favor extroversion can be detrimental if those practices are not what are actually preferred by the team members.
Does this matter? Is there actually a problem? What impact might this have?
Let's look at common complaints that we have heard from programmers who work on Agile teams.
“I can't focus! There are too many distractions and disruptions.”
“Too many meetings!”
“I don't actually
want
to know what everyone else is working on!”
“I don't actually
want
to self-organize! I want someone to
get us organized
so that I can code! I just want them to let me
code the way that I want to
.”
“I don't actually
want
to reach out to others for ad hoc discussions! I only do that as a last resort if I am really stuck.”
“Not enough attention is paid to technical issues!”
If these complaints are valid, it is a pretty bad state of affairs, because the true needs of programming teams are not being addressed by the Agile community; in fact, it is worse than that: programmers are being literally coerced, by organizations that adopt Scrum as a mandatory methodology,22 into working in a way that is not suited to their personalities.
Coaches Should Not Assume
What has continued to amaze us is that if one asks an Agile coach, “Is Agile working?” the answer will usually be an emphatic yes followed by gushing about how productive teams are and how Agile has revolutionized software development. Yet if one asks the typical programmer—the very people who Agile was created for—they will have a different range of responses. In fact, it seems to us that most programmers have not had a very positive experience with Agile.
Studies at the University of Florida and the University of Notre Dame indicate that introverted team members tend to view their extroverted co-workers more negatively, whereas the extroverted co-workers do not seem to have that negative bias.23 In other words, the introverts silently judged the extroverts negatively.
We think something like that might be happening among Agile teams. Agile coaches tend to be people-oriented, and therefore more extroverted than programmers on average, so they don't see a problem; programmers tend to be more introverted than average.24 The Agile coaches think things are great. The programmers? Well, not so much.
The University of Florida and Notre Dame experiments might also indicate that extroverts tend to be relatively insensitive to how introverts are experiencing things, because if they were aware, they would probably not be so favorable in their own ratings of their introverted colleagues. That conclusion is also supported by a Yale study that indicated the introverts are generally more sensitive to the feelings of others.25
This is why coaches should not assume that what they would prefer is what their team will prefer. Ask! And pay attention to what is being written and said, even if it goes against Agile dogma. For example, in her book Carved In Sand, Cathryn Ramin writes the following on page 33:
“… an endless stream of interruptions has become the norm. Researchers at the University of California, Irvine attempted to quantify the number of distractions and interruptions that occur among IT workers in a medium-sized office. They predicted that something would interfere with concentration every fifteen minutes, but on average, interruptions occurred every three minutes, and only two-thirds of the interrupted work was resumed on the same day.”
If this sounds like your organization, do you think that team members can focus? Ask them!
