31,19 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
This book will help you overcome the common challenges you’ll face when transforming your working practices from waterfall to Agile. Each chapter builds on the last, starting with easy-to-grasp ways to get going with Agile. Next you’ll see how to choose the right Agile framework for your organization. Moving on, you’ll implement systematic product delivery and measure and report progress with visualization. Then you’ll learn how to create high performing teams, develop people in Agile, manage in Agile, and perform distributed Agile and collaborative governance.
At the end of the book, you’ll discover how Agile will help your company progressively deliver software to customers, increase customer satisfaction, and improve the level of efficiency in software development teams.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 519
Veröffentlichungsjahr: 2018
Ähnliche
The Agile Developer's Handbook
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Amarabha BanerjeeAcquisition Editor: Namrata Patil Content Development Editor: Deepti ThoreTechnical Editor: Sneha HanchateCopy Editor: Safis Editing, Laxmi SubramanianProject Coordinator: Shweta BirwatkarProofreader: Safis EditingIndexer: Aishwarya GangawaneGraphics: Tania DuttaProduction Coordinator: Melywn Dsa
First published: February 2018
Production reference: 2090518
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78728-020-5
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Paul Flewelling began looking for a better way to build software after one fateful project that ticked all the boxes, yet failed to deliver what the customer needed. That search led him to help grow one of NZ's first Agile software teams. As word of the team's success spread, he also began training others. He currently coaches the product teams who are transforming NZ's largest newspaper company for the digital economy. He speaks nationally and internationally on all things Agile, shares coaching insights on his blog, co-hosts monthly meet-ups, and is often seen wisely stroking his beard.
I want to thank all the teams I've coached for coming on the journey with me. Your willingness to try out many of our crazy ideas and make them work inspires me to be a better coach.
Thank you to the many coaches I've worked and shared ideas with. You've always stretched me to think beyond my horizons; I'm humbled to work with such great humans.
Finally, I want to express my gratitude to my wife; I wouldn't be on this journey if it weren't for her.
About the reviewer
Kamal Prasad Bansod has around 15 yrs of experience in IT. He has spent around 8 years exploring and experimenting with Agile/Scrum. He has worked with different team dynamics and transformation of Agile teams. He strongly believes in the Agile principles and engineering practices. If they are followed properly then value-based delivery can be done to the market with expected quality. He loves to share his knowledge and ideas with companies that adopt and improve their use of Agile processes and techniques to build high-performing teams.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
The Agile Developer's Handbook
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the color images
Conventions used
Get in touch
Reviews
The Software Industry and the Agile Manifesto
Why the software industry needed to change
Delivery as a software product
Delivery as a software project
Product versus project
Scope was the priority
Estimates
Uncertainty buffers
Estimates became ironic
Variability in our estimates
And then there's missing the point entirely
Where's the business value?
So the project mindset isn't good?
The Agile values
The Agile principles
Incremental – adaptive versus waterfall – predictive
The Waterfall process and predictive planning
Incremental delivery and adaptive planning
Agile is a mindset
An Example of "Being Agile"
Scenario
Our response
Summary
Agile Software Delivery Methods and How They Fit the Manifesto
Understanding Scrum
Background
Introduction to the mechanics of Scrum
Sprint Planning – part 1
Sprint Planning - part 2
The Daily Scrum
The Sprint Review
The Sprint Retrospective
Additional events
XP - Extreme Programming
Background
Introduction to the mechanics of XP
The planning game
Part 1 – Release planning
Part 2 – Iteration planning
Implementing the iteration plan
Iteration demo
Iteration retrospective
Kanban and Lean Software Development
Background
Reducing waste
Single-piece flow
How Kanban/Lean fit into the Agile movement
Introduction to the mechanics of Lean/Kanban
Getting started with Kanban
Step 1 – Make the team's work visible
Step 2 – Make the work policies of the team explicit
Step 3 – Improve flow
Step 4 – Kaizen or continuous improvement
Choosing the right framework
Designed for small teams
They don't include product discovery phases
Not all frameworks prescribe technical practices
There are similarities with subtle differences
Mixing and matching Agile methods
Summary
Introducing Scrum to your Software Team
Why Scrum is an excellent place to start
Iterations and iteration length
Starting a new Scrum team
Prerequisites
Preparing to Sprint
Activity – defining the Product Backlog
Activity – release planning
Activity – introducing the Product Backlog
Activity – estimating User Stories on the backlog
Activity – setting up the Scrum Board
Discussion – Sprint Zero
Day one of the Sprint
Event – Sprint Planning
What can we achieve in this Sprint?
How will we achieve it?
The Sprint Goal
Event – first Daily Scrum
A day in the life of a Scrum team
Measuring and reporting progress with visualization
Avatars
Done stickers
Burndowns
Sprint Goal confidence
The importance of visible workspaces
Removing impediments
Managing delivery as a team sport
The last day of the Sprint
Event – Sprint Review
Event – Sprint Retrospective
The importance of team retrospectives
Summary
Gathering Agile User Requirements
The pitfalls of traditional big upfront requirements
Why User Stories produce results
What a User Story is and how to use it
The User Story format
Additional story card elements
The stub
Acceptance criteria
Story size
Notes
Card, conversation, confirmation
Acceptance criteria versus the Definition of Done (DoD)
Telling the team "what", not "how"
Brainstorming a bunch of User Stories
Estimating Agile user requirements
Planning Poker
Playing Planning Poker
References
Summary
Bootstrap Teams with Liftoffs
What's a team liftoff?
ACTIVITY – Agile training
ACTIVITY – Sharing the vision
Step 1 – Meet the sponsors
Step 2 – The company purpose
Step 3 – The product vision
Step 4 – The current mission
Step 5 – Define success
Defining success metrics
Activity – Forming a team charter
Step 1 – Defining done
Step 2 – Working agreement
Step 3 – Team calendar
References
Summary
Metrics that will Help your Software Team Deliver
A brief introduction to measurements for Agile software delivery
Understanding measurements
Qualitative versus quantitative measurements
Negative versus positive metrics
Quantitative metrics
Team velocity
Sprint Burndown chart – TEAM
Release burndown charts
Simple Release Burndown chart
Enhanced Release Burndown chart
Code quality
Code complexity
Qualitative metrics
Defining what success looks like
Defining our success
Using our Team Success Indicators
User Happiness Index
Summary
Software Technical Practices are the Foundation of Incremental Software Delivery
Building the thing right versus building the right thing
Refactoring
How does this keep us Agile?
Things to try
Test-Driven Development
How does this keep us Agile?
Things to try
Pair programming
How does this keep us Agile?
Things to try
Activity – pair programming ping pong
Emergent design
How does this keep us Agile?
Things to try
Activity – emergent design discussion
The DevOps culture
Continuous Integration
How does this keep us Agile?
Things to try
Continuous Delivery
How does this keep us Agile?
Things to try
Continuous Deployment
How does this keep us Agile?
Things to try
Summary
Tightening Feedback Loops in the Software Development Life Cycle
Implementing incremental delivery in Agile
Working with software in small, manageable chunks
Inspection and adaption
The importance of User Experience (UX)
Shifting left
Shifting right
Introducing some Lean thinking to improve flow
The coin game results
Systems thinking – Optimizing the whole
Changing our workflow
Kaizen and developing a small, continuous improvement mindset
Fail Cake
Root cause analysis with the Five Whys method
Adopting Lean Startup methods to validate ideas
Build, Measure, Learn
An example of Lean Startup MVP
Background
Hypothesis
Approach
Learning rapidly by doing and failing fast
Summary
Seeking Value – How to Deliver Better Software Sooner
Moving from project to product thinking
Cynefin – A sense-making framework
The advantages of a product team
Moving from project to product
Setting objectives to create alignment of purpose
Using Objectives and Key Results (OKRs)
Hypothesis-Driven Development (HDD)
Data, Insights, Beliefs, Bets (DIBBs)
Seeking value
How to seek value
Telling our team "the why" not "the what"
Summary
Using Product Roadmaps to Guide Software Delivery
The importance of Product Roadmaps
Product Discovery to create roadmaps
User Story Mapping
Activity – creating a User Story Map
Step 1 – User roles
Step 2 – The first User Journey
Step 3 – Alternative paths, sub-tasks, details, and business rules
Step 4 – Naming the activities
Leveraging the User Story Map
Impact Mapping
Activity – Creating an Impact Map
Question 1 – Why are we doing this?
Question 2 – Who will help us?
Question 3 – How will they help us?
Question 4 – What will we do?
Leveraging an Impact Map
Working collaboratively with the team to create a shared understanding
Using Spikes to reduce business and technical unknowns
Using Rolling Wave Planning for adaptive delivery
Summary
Improving Our Team Dynamics to Increase Our Agility
How to create high-performing teams
Collaboration is the key
Knowledge work and high-bandwidth communication
Communication lines
Psychological safety – what it is and how to create it
Further reading
The stages of team formation
Stage 1 – forming
Helping our team process this phase
Relationship-building games
Stage 2 – storming
Helping our team process this phase
Coach – diversity is a good thing
Coach – conflict happens, how to navigate it
Positivity and successful relationships
Activity – process the conflict
Group decision making
Coach – focus on the task, not the person
Coach – conflict protocols
Coach – giving feedback
Stage 3 – norming
Helping our team process this phase
Activity – experiments in process
Activity – improve the team's ability to self-assess
Stage 4 – performing
Helping our team process this phase
Activity – 360 team review
Stage 5 – adjourning/mourning
Helping our team process this phase
Learning by practicing
Hackathons
Innovation Days
Running a hackathon or innovation day
Google's 20% time and other alternatives
Summary
Baking Quality into Our Software Delivery
Cross-pollination of skills
Mob programming
Why does mob programming work?
How does mob programming work?
Ergonomics
No more bugs
Bugs caused by the way we write code and design our software
Bugs caused by our understanding of the problem
Bugs caused by the nature of the problem changing
Refactoring our environment
No more estimates
Summary
The Ultimate Software Team Member
The power of motivation
Extrinsic drivers
Intrinsic drivers
Mastery
Autonomy
Purpose
Activity – moving motivators
Why are intrinsic drivers important?
The entrepreneurial developer
The team player
The problem with "rockstars"
The growth mindset
Growth mindset versus fixed mindset
An example of a fixed mindset behavior versus a growth mindset behavior
How can we embrace a growth mindset?
Fostering a learning habit
Summary
Moving Beyond Isolated Agile Teams
How an Agile approach will organically flatten an organizational structure
From a hierarchical organization to collaborative governance
The Big Agile board
A network of teams
Self-selection to create a true-self organization
Further reading
Modern leadership
Changing to a fly-by-wire system
Implementing a fly-by-wire approach
Setting the challenge
Cultivating culture
Ensuring team engagement
Feeling recognized
Feeling relevant
Feeling successful
Creating engagement
Example – Changing up the performance review
Performance review 360 – Intermediate level
Performance Review 360 – Advanced level
The art of Agile leadership
Experiment – Creating informal networks as a nervous system
Experiment – Communities
Organizational styles for flatter structures
Further reading
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Agile thinking has caused a fundamental shift in how we deliver software. The aim of this book is to offer the reader a pragmatic, easy-to-follow approach to mastering this transformative way of working. It's roughly divided into three sections to reflect the three stages of adoption—beginner, intermediate, and master.
In the first few chapters, we'll look at the origins of the Agile movement and review the fundamentals. We then get straight into the practical guide, teaching you how to set up your Agile approach through a series of activities and pragmatic examples. Each key concept has an accompanying exercise for you to practice and learn the thinking behind it.
After we've laid the ground rules and taught the basic techniques, the second half of the book will look at how we can start to expand our understanding. It takes what we've learned and applies slightly different perspectives, such as Lean and Lean Startup thinking. You'll start to see how you can shape or bend the rules to get much more from your Agile practice.
In the final three chapters, we look at some possibilities regarding technical practices, team dynamics, and organizational changes. Here, the intention is to show you where some teams are now. And although some of these may seem distant from where you are on your particular journey, the aim is to show where you may be heading and to inspire you to keep taking the next step.
Agile is a mindset, something we can only achieve by going on the journey ourselves. The hope is that this book will put you on the path to becoming a truly Agile thinker.
Let's go!
Who this book is for
This is a companion guide for those who build software and are about to embark on (or who are already on) an Agile journey. No prior knowledge of Agile is needed. This book fills in the gaps. For those at the very beginning of their journey, this book teaches the foundations to start your Agile practice. For those who are already on the path, this book is designed to give you ideas on where you may be heading. Like any good guide, it gives tips to move you out of your comfort zone and push your thinking to become truly Agile.
What this book covers
Chapter 1, The Software Industry and the Agile Manifesto, states that a crisis in the software industry led to a major turning point and the formation of the Agile Alliance. We'll examine the Alliance's origins and discuss how it revolutionized the way we think about building software. We'll also look at the Agile Manifesto, its values and principles, and the impact they have on how software professionals work.
Chapter 2, Agile Software Delivery Methods and How They Fit the Manifesto, provides an overview of the mechanics and the rationale behind the three most popular Agile methods: Scrum, XP, and Kanban. We'll look at their similarities and differences, as well as how they look to achieve the Agile Manifesto's guiding values and principles.
Chapter 3, Introducing Scrum to your Software Team, provides a step-by-step guide to your first Scrum Sprint. We'll walk you through all of the key activities, including setting up your Agile board, planning to Sprint, a day-in-the-life of a Scrum team, and the differences between the Sprint Review and the Sprint Retrospective. The aim is to give you everything you need to take the first step on your Agile journey.
Chapter 4, Gathering Agile User Requirements, explains that working iteratively to incrementally deliver software needs a different approach to requirements gathering. We look at the approach most commonly used by Agile methods, the User Story. We'll look at how to write a well-defined story. We'll introduce Agile estimation using a technique called relative sizing, and we'll play our first game of planning poker.
Chapter 5, Bootstrap Teams with Liftoffs, explains that the aim of a liftoff is to set teams up for the best possible chance of success. We'll walk you through the activities of a liftoff, which include communicating the vision, defining mission outcomes, and deciding how you're going to work together.
Chapter 6, Metrics that will Help your Software Team Deliver, states that once your team is on a mission, it's important that you know you're moving in the right direction. Taking the right measurements will help us do that. We'll look at various real-world examples that will help your team determine the metrics they should and shouldn't be using. We'll look at negative metrics, positive metrics, metrics that give you quantitative feedback and metrics that give you qualitative feedback. We'll then discuss how to make the information visible and measure the trends over time so that you can see you're improving and moving along the right trajectory.
Chapter 7, Software Technical Practices are the Foundation of Incremental Software Delivery, explains that the right technical practices will increase your team's agility. Practices such as Refactoring, Test-Driven Development, Pair Programming, and Continuous Integration give us confidence that what we're delivering is well designed, tested, and meets expectations. We'll look at these practices, as well as others, and offer ideas on how to introduce them to your team's toolkit.
Chapter 8, Tightening Feedback Loops in the Software Development Life Cycle, shows how to implement an incremental approach to software delivery. We discuss the important role that user experience plays in helping us tease out what our customer needs. We introduce Lean thinking and show how it seeks to optimize our process so that we can deliver sooner. We also show how to shortcut our learning by conducting a Lean Startup experiment with a real-world example.
Chapter 9, Seeking Value – How to Deliver Better Software Sooner, looks at real-world examples to shift the delivery team's perspective from one of just delivering on time, on scope, or on budget, to ways that will actually seek value. We also look at the practical ways teams can use this approach to create feedback loops so that they can measure their value-seeking success as they go.
Chapter 10, Using Product Roadmaps to Guide Software Delivery, explains that one of the roles of the Product Owner is to hold the product vision for the team. We show how to discover our key product features using two different techniques: User Story Mapping and Impact Mapping. We show how to prioritize features/User Stories against release milestones to create the initial Product Roadmap. We also introduce Rolling Wave Planning, a technique for adaptive planning.
Chapter 11, Improving Our Team Dynamics to Increase Our Agility, states that Agile teams are self-organized, autonomous, and prefer high-bandwidth face-to-face communication. This may seem like a recipe for a high performing, collaborative team, but it doesn't always work that way. This chapter aims to demystify some of the art of fostering good team dynamics by looking at what makes a great team. We also describe the five stages of team formation and give techniques for helping a team become the best it can be.
Chapter 12, Baking Quality into Our Software Delivery, looks at several popular approaches that teams take to increase their performance further. We discuss the cross-pollination of skills in cross-functional teams, a total team approach called mob programming, how to live the dream of having no more bugs, and we question the need for estimates in software development.
Chapter 13, The Ultimate Software Team Member, speaks about the power of motivation and how to tap into our inner drive to become the best at what we do. We define some of the characteristics of a great Agile team member. We also consider the fact that, as knowledge workers, we're in a constant state of learning, and give some pragmatic suggestions for putting time aside to learn.
Chapter 14, Moving Beyond Isolated Agile Teams, explains that in order to become an Agile organization, a change in management style is needed. We look at how the leadership model begins to switch to a supportive role and introduce the concept of the Servant Leader. We look at how modern leadership and different organizational structures create an adaptive, more responsive network of teams. This chapter will help you recognize the signs that an organizational transformation is taking place so that you can take advantage of this transformative approach to work.
To get the most out of this book
No software or hardware is needed; just bring your brain and we'll do the rest. All of the activities we describe are easy for you to replicate yourselves, using everyday materials such as coins, spaghetti, and marshmallows.
To create your visible workspaces and run a few of the activities, you will need the following in your Agile toolkit: a near unlimited supply of Post-it notes, Sharpies (black markers), index cards, a whiteboard and whiteboard markers, and maybe sticky tape for marking areas on your board.
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/TheAgileDevelopersHandbook_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
The Software Industry and the Agile Manifesto
Towards the end of the last century, the software industry was in a state of turmoil. There were some significant software project failures, which were having a substantial reputational impact on the industry as a whole. One observation was that software projects over a certain size were more likely to fail.
In this chapter, we'll discuss the factors that led to a crisis and the subsequent major turning-point in the software industry. In the second part of this chapter, we'll introduce the Agile Manifesto and its origins and discuss the way it revolutionized the way we think about building software. We'll explain the Manifesto's values and principles, what they each mean, and the impact they have on how software professionals work.
In this chapter, we will cover the following topics:
Why the software industry needed to change
The origins of the Agile Manifesto
A detailed look at the values and principles of the manifesto and how they translate to today's context
Adaptive versus predictive planning
Incremental versus waterfall delivery
Agile isn't a process, it's a mindset built on guiding values/principles and requires solid technical practices
Why the software industry needed to change
When I was outlining this book, I was in two minds about whether to include a section on the history of the software industry and how Agile came to be. I figured most of you would be young enough not to know any different; you're probably doing something Agile or closely related to Agile already. However, I decided to include this chapter for two reasons:
I still see echoes of the past affecting us today
"Why" we do something is important to inform "what" we do and "how" we do it
When working as a software developer in the 1990s and 2000s, I worked in a range of different organization types, from software houses to software consultants, from large organizations such as corporate and central government to small three-person startups. In my experience across these different organization types, we used two noticeably different styles of software delivery, either as a product or a project.
Delivery as a software product
When building software products, we tended to form long-lived teams around them. The team would be responsible for the product throughout its life. During the initial build and subsequent revisions, they handled the entire Software Development Life Cycle (SDLC) end-to-end, including delivery into production. These teams were also often responsible for managing the production environment and providing support.
The scale of the product, and how widely it was adopted, determined whether it was managed by a single software team or by a network of teams all working for one product group. One thing that was noticeable about software product teams, apart from their long-lived nature, was that they often had good relationships with their customers as well as technology and operations staff. Sometimes they even had representatives of these groups within the teams.
Funding these products often took place on a yearly cycle, which coincided with setting the objectives for the product's development that year. A key thing to note is that the budget would often be allocated to fund the team(s) and the decisions on what features would be developed, enhanced, or fixed that year would be managed by the team itself.
Delivery as a software project
When treating software delivery as a project, the approach was often very different. The project team would only form for the duration of the build of the software.
Once built, the software would often be handed over to a separate team, known as the Business As Usual (BAU) team in business parlance. They would manage the maintenance and support of the product. There was a two-fold intention in handing over to a separate BAU team:
They would handle the bulk of changes to the organization, for example, by training all impacted staff on how to use the new software and associated business processes. Once they'd introduced the changes, the BAU team's aim would then be to create and support a stable business environment.
The software delivery team would be free to move on to the next project. Software delivery teams were a scarce resource, and this was seen as a way to optimize software delivery.
Software projects required project managers who often operated in the change management capacity as well, although sometimes separate change managers were allocated. In this way, they would also be responsible for seeing that the introduction of the new software platform to the organization and the transition to BAU went smoothly. The project team itself would often be managed by a separate unit who reported directly to the business known as the Project Management Office (PMO).
Product versus project
What I discovered by using these two contrasting approaches resulted in a profound shift in my thinking: When delivering software as a product, there was much more opportunity to focus on value because the product team was long-lived and was able to adopt an iterative/incremental approach to delivery. We, therefore, had multiple opportunities to deliver, enhance our strategy, and get things right.
However, with software delivery as a project, more often than not the software delivery team only had one opportunity to get it right. Successfully delivering a software project to meet expectations with only one shot just didn't happen that often.
Even when we did deliver, it was often as a result of a considerable effort on our part to get it across the line, including many lost evenings and weekends.
Subsequent revisions of the software were often handled as separate projects and likely by a different software team, often leading to a lack of continuity and knowledge sharing.
As a result, there was a distinct lack of trust between us—the software professionals and the people we were building software for. Unfortunately, "us" often became them and us with unmet expectations being so often the cause of the rift in the relationship.
We tried to solve this problem by making things more precise. Unfortunately, the version of precision the project mindset opted for had only three predictive aspects—scope, time, and budget. All three of these things are very difficult to quantify when tied to a software project where complexity, uncertainty, and sheer volume of work could and did amplify any errors in these calculations.
However, the single most significant problem when you tie down all three of these factors, scope, time and budget, is that something colloquially known as the Iron Triangle forms. Refer to the following figure:
When you set scope, date, and budget like that, there is little room for maneuverability to deviate from the plan. To help mitigate risks, most will create buffers in their schedules. However, the rigid nature of the triangle means that if and when overruns start to eat more and more into the buffers, something else has to give. And what usually occurs when a software development team is under pressure to deliver? One or more of the following qualities of your software product will start to suffer:
Functionality
: Whether it works as expected
Reliability
: How available it is
Usability
: How intuitive it is to use
Scalability
: How performant it is
Maintainability
: How easy it is to change
Portability
: How easy it is to move to a different platform
To understand why precision in predicting the outcome of a software project is so complicated we need to unpack things a little.
Scope was the priority
At the time, many felt that the functionality of the delivered software was the priority and would often seek to lock the scope of the project. We allowed for this by having some degree of variability in the budget and the schedule.
At the time, many project managers would work to the PMI or ISO 9000 guidelines on the definition of quality. Both of these had a reasonably straightforward quality definition requiring the scope of the project to be delivered in a fully functional form.
To meet the scope expectation, we had to estimate, to a fair degree of precision, how long it would take us. In this way, we would be able to determine the length of time needed and the number of people required for the team.
And it was at the estimate stage that we often set ourselves up to fail.
Estimates
One thing that was increasingly obvious in the software industry was that our approaches to estimating or predicting the outcome of a project were out of whack. As a software team, we were often passed detailed requirements to analyze, then do some initial design work, and provide a work breakdown with estimates.
We'd be working on one project and then at the same time asked to estimate on another. The estimates we were told would inform the decision on whether the project was viable and would be funded. Some teams tried to be sophisticated, using algorithmic estimating techniques such as Constructive Cost Model (COCOMO), COCOMO II, Source lines of code (SLOC), and Function Point. Most estimating methods incorporated rule of thumb or experience-based factors as well as factors for complexity and certainty.
Sometimes the estimates would be given by people who weren't going to work on the project. This was either because the project leadership group didn't want to disturb an existing project team, or because, without funding, the project teams weren't formed yet. This meant the involvement of someone like an architect or a technical leader who could break down the work and estimate based on the solution they devised. Most complicated problems have several solutions, so if those who had done the solution work breakdown weren't available to provide guidance on how to implement their solution, obviously this could cause trouble later on.
Either way, whoever provided the estimate would give it based on the best solution they could theorize with the information they were given. More often than not, the first estimate that was given would be used as a way to control the project, and it would be pretty much set in stone. This was a pattern that I stumbled across many times. When this continually happened, we tried to improve the accuracy of our estimates by spending time doing more upfront work. But the reality was to get an exact estimate of effort so that time frames and budget can be drawn, you pretty much have to complete all the work in the first place.
Uncertainty buffers
It also became a bit of a standing joke that these estimates were not only painstakingly worked on over days or weeks, but as soon as we gave them to a project manager, they would double the figures. When we challenged that practice, they'd remind us that all software developers are optimistic and that buffers were needed.
But it's not that we're optimistic by nature; in fact, a lot of us had already factored in our own buffer allowances. The explanation is more straightforward; the work we do is novel—more often than not we are working on something entirely different from what we've built before: different domain, different technology stack, different frameworks, and so on.
Some teams would combat this by offering two figures as part of their estimates. The first being the time estimate, the second being the level of certainty that they could complete it within that timeframe. If the task was straightforward, then they would offer their estimate with 100% certainty. If the task was more complicated they would lower the percentage accordingly. The higher the uncertainty, the lower the percentage.
This certainty factor could then be used to allocate a buffer. As certainty got higher, the buffer would get smaller, but even with 100% certainty, there would still be a buffer. After all, there is no such thing as an ideal day as much as we would like to think there is.
At the opposite end of the spectrum, the more the uncertainty in the estimate, the larger the buffer. At the extreme, it was not uncommon to have buffers of 200%.
Estimates became ironic
The chronic misestimation of development effort has led to some Dilbert-esque observations of the way we work.
For example, we would often refer ironically to a task as just a "small matter of programming," when someone with little or no understanding of what was involved was telling us it looked easy.
We also developed ironic laws about underestimation such as the Ninety-Ninety rule, which states the following:
This rule was later made popular by Jon Bentley's September 1985 Programming Pearls column in Communications of the ACM, where it is titled "Rule of Credibility".
Variability in our estimates
The biggest problem with estimation is the amount of information we assume. We make assumptions on how to solve the business problem, the technologies we're going to use, and the capabilities of the people building it. So many factors.
The level of complexity and uncertainty impacts our ability to give an accurate estimate because there are so many variables at play. This, in turn, is amplified by the size of the piece of work. The result is something referred to as the Cone of Uncertainty:
Barry Boehm first described this concept in his book Software Engineering Economics, 1981; he called it the Funnel Curve. It was named The Cone of Uncertainty in the Software Project Survival Guide (McConnell 1997).
It shows us that the further we are away from completion, the larger the variance in the estimate we give. As we move closer to completion, the more accurate our estimate will become, to the point where we complete the work and know exactly how long it took us.
So while it was felt that better precision could be gained using a gated process, such as Waterfall, because it led to a tendency to bundle more of the stuff we wanted to get done together, it would significantly increase the size of the work parcel. This, in turn, compounded the problem of getting an accurate estimate.
And then there's missing the point entirely
Sometimes, of course, we'd fail to deliver something of use to our customer and miss the point entirely.
I remember one project I worked on where we were given ideal working conditions. The teams were offsite, in our own offices, which meant we could be dedicated to the project we were working on without being disturbed or shoulder-tapped by others. It felt like the perfect setup for success.
We spent ten months painstakingly delivering precisely to requirements. Everything was built and tested out according to the detailed designs we were given. We were even on budget and time when we delivered. Unfortunately, when the software went live and was in the hands of the people using it, they reported back that it didn't do the job they needed it to do.
Why had we failed? We'd failed because we spent ten months building something in isolation from our customer. We hadn't involved them in the implementation, and too many assumptions had been made. Diagrammatically, this looked a little like the following:
We then spent the next six months reworking the software into something that was usable. Unfortunately, for the partner company working alongside our team, this meant a major variance in their contract, most of which they had to swallow. We eventually did deliver something to our customer that they wanted and needed but at a huge financial impact on us and our partner company.
This, unfortunately, is the path that predictive planning sets you on. You develop a fixed mindset around what is to be delivered because you know if you aren't dogmatic in your approach, you're likely to fail to meet the date, budget, or scope set in the contract.
Where's the business value?
One of the key characteristics to understand about the nature of predictive planning is that the minute someone says, "How much is this going to cost?", or "When can this be delivered?", they significantly constrain the value that will be created. And, at the end of the day, shouldn't it be about maximizing the value to your customer? Imagine if we spent one more month on a project and delivered twice as much value. Wouldn't that be something our customer would want over meeting a date or a particular set of features just because it's been speculatively laid out in a contract?
Instead, we focus on a date, an amount of money, and the functionality. And unfortunately, functionality doesn't always translate to value (as we saw in the previous section when we missed the point entirely).
This is why the predictive planning used in Waterfall style deliveries has also become known as faith-driven development because it leaves so much to chance, and usually right until the end of the project.
To focus on value delivery, we have to shift our mindset to use adaptive planning versus predictive planning, something that we will talk about later in this chapter.
So the project mindset isn't good?
It's not that the project mindset is bad, it's just that the mindset drives us to think that we need a big upfront design approach in order to obtain a precision estimate. As noted, this can lead to a number of issues, particularly when presented with a large chunk of work.
And it isn't that upfront design is bad; it's often needed. It's just the big part, when we try to do too much of it, that causes us problems.
There were a number of bad behaviors feeding the big thinking happening in the industry. One of these is the way that work is funded, as projects. For a particular project to get funded, it has to demonstrate its viability at the annual funding round.
Unfortunately, once people in the management seats saw a hard number, it often became set in stone as an expectation. During the execution of the plan, if new information was discovered that was anything more significant than a small variance, there would be a tendency to try to avoid doing it. The preference would be to try to stick to the plan, rather than incorporate the change. You were seen as a better project manager for doing that.
So, we had a chicken-and-egg scenario; the project approach to funding meant that:
The business needed to know the cost of something so they could allocate a budget.
We needed to know the size of something and its technical scope and nature so that we could allocate the right team in terms of size and skill set.
We had to do this while accepting:
That our business didn't know exactly what it needed. The nature of software is intangible, most people don't know what they want/need until they see it and use it.
The business itself wasn't stationary; just because we recorded requirements at a particular moment in time, didn't mean that the business would stop evolving around us.
So, if we can avoid the big part, we're able to reduce the level of uncertainty and subsequent variability in our estimates. We're also able to deliver in a timely fashion, which means there is less likelihood of requirements going out of date or the business changing its mind.
We did try to remedy this by moving to prototyping approaches. This enabled us to make iterative sweeps through the work, refining it with people who could actually use the working prototype and give us direct feedback.
Rapid Application Development, or RAD as it's commonly known, is one example of an early iterative process. Another was the Rational Unified Process (RUP).
Working on the principle that many people didn't know what they wanted until they saw it, we used RAD tools such as Visual Basic/Visual Studio to put semi-working prototypes together quickly.
But we still always managed to bite off more than we could chew. I suspect the main reason for this is that our customers still expected us to deliver something they could use. While prototypes gave the appearance of doing that and did enable us to get feedback early:
At the end of the session, after getting the feedback we needed, and much to our customer's disappointment, we'd take the prototype away. They had assumed our mockup was a working software.
We still hadn't delivered anything they could use in their day-to-day life to help them solve real-world problems.
To try to find a remedy to the hit-and-miss approach to software delivery, a group of 17 software luminaries came together in February 2001. The venue was a cabin in Utah, which they chose, as the story goes, so that they could ski, eat, and look for an alternative to the heavyweight, document-driven processes that seemed to dominate the industry.
Among them were representatives from Extreme Programming (XP), Scrum, Dynamic Systems Development Method (DSDM), Adaptive Software Development (ASD), Crystal, Feature-Driven Development, and Pragmatic Programming.
While at least one of them commented that they didn't expect anything substantive to come out of that weekend, what they did in fact formulate was the manifesto for Agile software development.
The manifesto documents four values and twelve principles that uncover "better ways of developing software by doing it and helping others do it".
The group formally signed the Manifesto and named themselves the Agile Alliance. We'll take a look at the Agile Values and Principles in the following sections.
The Agile values
Here is the Manifesto for Agile Software Development (http://agilemanifesto.org/):
To understand the four values, you have first to read and understand the end subtext:
That is, while there is value in the items on the right, we value the items on the left more.
Let's look at how this works by looking at each value in more detail:
Individuals and interactions over processes and tools
: In an Agile environment, we still have processes and tools, but we prefer to keep our use of them light, because we value communication between individuals. If we're to foster successful collaboration, we need common understanding between technical and non-technical people. Tools and processes have a tendency to obfuscate that.
A good example is the User Story, an Agile requirement gathering technique, usually recorded on an index card. It's kept deliberately small so that we can't add too much detail. The aim is to encourage, through conversation, a shared understanding of the task.
In the same way, we should look at all of the following Agile values:
Working software over comprehensive documentation
: As a software delivery team, our primary focus should be on delivering the software—fit for purpose, and satisfying our customer's need.
In the past, we've made the mistake of using documents to communicate to our customer what we're building. Of course, this led to much confusion and potential ambiguity. Our customer isn't an expert in building software and would, therefore, find it pretty hard to interpret our documentation and imagine what we might be building. The easiest way to communicate with them is via working software that they can interact with and use.
By getting something useful in front of our customer as soon as possible, we might discover if we're thinking what they're thinking. In this way, we can build out software incrementally while validating early and often with our customer that we're building the right thing.
Customer collaboration over contract negotiation
: We aim to build something useful for our customer and hopefully get the best value for them we can. Contracts can constrain this, especially when you start to test the assumptions that were made when the contract was drawn up. More often than not there are discoveries made along the way, or the realization that something was forgotten or that it won't work the way we were expecting. Having to renegotiate a contract, or worse still, recording variances to be carried out at a later stage, both slow down and constrain the team's ability to deliver something of value to the customer.
Responding to change over following a plan
: When considering this Agile Value, it is worth drawing a comparison with the military.
The military operates in a very fluid environment; while they will undoubtedly have a plan of attack, this is often based on incomplete information about the enemy's strength and whereabouts. The military very much has to deal with known knowns, known unknowns, and unknown unknowns.
This is what we call a planning-driven environment; they're planning constantly throughout the battle as new information becomes available.
So when going into battle, while they have group objectives, the military operate with a devolved power structure and delegated authority so that each unit can make decisions on the ground as new information is uncovered. In this way, they can respond to new information affecting the parameters of their mission, while still getting on with their overall objective. If the scope of their mission changes beyond recognition, they can use their chain of command to determine how they should proceed and re-plan if necessary.
In the same way, when we're building software, we don't want to blindly stick to a plan if the scope of our mission starts to change. The ability to respond to new information is what gives us our agility; sometimes we have to deviate from the plan to achieve the overall objective. This enables us to maximize the value delivered to our customer.
The Agile principles
The signatories to the Manifesto all shared a common background in light software development methodologies. The principles they chose reflect this. Again the emphasis is on people-focused outcomes. Each of the following principles supports and elaborates upon the values:
Our highest priority is to satisfy the customer through the early and continuous delivery of valuable software
: In encouraging incremental delivery as soon and often as we can, we can start to confirm that we are building the right thing. Most people don't know what they want until they see it, and in my experience, use it. Taking this approach garners early feedback and significantly reduces any risk to our customer.
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage
: Instead of locking scope and ignoring evolving business needs, adapt to new discoveries and re-prioritize work to deliver the most value possible for your customer. Imagine a game of soccer where the goal posts keep moving; instead of trying to stop them moving, change the way you play.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale
: The sooner we deliver, the sooner we get feedback. Not only from our customer that we're building the right thing, but also from our system that we're building it right. Once we get an end-to-end delivery taking place, we can start to iron out problems in our integration and deployment processes.
Business people and developers must work together daily throughout the project
: To get a good outcome, the customer needs to invest in the building of the software as much as the development team. One of the worst things you can hear from your customer as a software developer is, "You're the expert, you build it." It means that they are about to have very little involvement in the process of creating their software. And yes, while software developers are the experts at building software, and have a neat bunch of processes and tools that do just that, we're not the expert in our customer's domain and we're certainly not able to get inside their heads to truly understand what they need. The closer the customer works with the team, the better the result.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done
: A software development team is a well-educated bunch of problem solvers. We don't want to constrain them by telling them how to do their jobs; the people closest to solving the problem will get the best results. Even the military delegate authority to the people on the frontline because they know if the objective is clear, those people are the ones who can and will get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation
: Face-to-face conversation is a high-bandwidth activity that not only includes words but facial expressions and body language too. It's the fastest way to get information from one human being to another. It's an interactive process that can be used to quickly resolve any ambiguity via questioning. Couple face-to-face conversation with a whiteboard, and you have a powerhouse of understanding between two or more individuals. All other forms of communication dwindle in comparison.
Working software is the primary measure of progress
:
When you think about a software delivery team, and what they are there to do, then there really is nothing else to measure their progress.
This principle gives us further guidance around the Agile value
working software over comprehensive documentation
.
The emphasis is on working software because we don't want to give any false indicators of progress. For example, if we deliver software that isn't fully tested, then we know that it isn't complete, it has to go through several cycles of testing and fixing. This hasn't moved us any closer to completion of that piece of work because it's still not done.Done is in the hands of our customer, done is doing the job it was intended to do. Until that point, we aren't 100% sure we've built the right thing, and until that moment we don't have a clear indication of what we might need to redo. Everything else the software team produces just supports the delivery of the software, from design documents to user guides.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely:
Putting a software delivery team under pressure to deliver happens all the time; it shouldn't, but it does. There are a number of consequences of doing this, some of which we discussed earlier in this chapter.
For example, put a team under pressure for long enough, and you'll seriously impact the quality of your product. The team will work long hours, make mistakes, take shortcuts, and so on to get things done for us. The result won't just affect quality, but also the morale of our team, and their productivity. I've seen this happen time and time again; it results in good people leaving along with all the knowledge they've accumulated.
This principle aims to avoid that scenario from happening. Which means that we have to be smart and use alternative ways of getting things done sooner. This means seeking value, ruthless prioritization, delivering working software, a focus on quality, and allowing teams to manage their work in progress so they can avoid multitasking.
Studies have shown that multitasking causes context switching time losses of up to 20%. When you think about it, when you're solving complex problems, the deeper you are into the problem, the longer it takes to regain context when you pick it back up. It's like playing and switching between multiple games of chess. It's not impossible, but it definitely adds time.
I've also seen multitasking defined as messing up multiple things at once.
Continuous attention to technical excellence and good design enhances agility
: By using solid technical practices and attention to detail when building software, we improve our ability to make enhancements and changes to our software.
For example, Test-Driven Development (TDD) is a practice which is as much about designing our software as it is testing it. It may seem counter-intuitive to use TDD at first, as we're investing time in a practice that seemingly adds to the development time initially. In the long term, however, the improved design of our software and the confidence it gives us to make subsequent changes enhances our agility.
