33,59 €
The AZ-400 Designing and Implementing Microsoft DevOps Solutions certification helps DevOps engineers and administrators get to grips with practices such as continuous integration and continuous delivery (CI/CD), containerization, and zero downtime deployments using Azure DevOps Services.
This new edition is updated with advanced topics such as site reliability engineering (SRE), continuous improvement, and planning your cloud transformation journey. The book begins with the basics of CI/CD and automated deployments, and then moves ahead to show you how to apply configuration management and Infrastructure as Code (IaC) along with managing databases in DevOps scenarios. As you make progress, you’ll explore fitting security and compliance with DevOps and find out how to instrument applications and gather metrics to understand application usage and user behavior. This book will also help you implement a container build strategy and manage Azure Kubernetes Services. Lastly, you’ll discover quick tips and tricks to confidently apply effective DevOps practices and learn to create your own Azure DevOps organization.
By the end of this DevOps book, you'll have gained the knowledge needed to ensure seamless application deployments and business continuity.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 633
Veröffentlichungsjahr: 2022
Prepare for the certification exam and successfully apply Azure DevOps strategies with practical labs
Subhajit Chatterjee
Swapneel Deshpande
Henry Been
Maik van der Gaag
BIRMINGHAM—MUMBAI
Copyright © 2022 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Rahul Nair
Publishing Product Manager: Niranjan Naikwadi
Content Development Editor: Nihar Kapadia
Technical Editor: Arjun Varma
Copy Editor: Safis Editing
Project Coordinator: Ashwin Kharwa
Proofreader: Safis Editing
Indexer: Tejal Daruwale Soni
Production Designer: Roshan Kawale
Marketing Coordinator: Nimisha Dua
First published: June 2020
Second edition: September 2022
Production reference: 3210922
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-80324-066-4
www.packt.com
Subhajit Chatterjee has a bachelor of engineering degree and a postgraduate diploma in information technology. He has also taken up many online certifications that have helped him learn and grow as a software engineering professional.
He has over two decades of experience in designing, implementing, and managing software development projects, using Microsoft and open source technologies. He is a seasoned engineering leader and has delivered many large and complex projects in Azure, IoT, enterprise integrations, web applications, and the mobility space.
He loves to solve problems and is passionate about sharing his learning with the community.
I want to thank the people who have been close to me and supported me all these years, especially my wife and my elder brother who have always been a source of motivation. I am also grateful to my work colleagues who are an inspiration themselves and have helped me learn and grow.
Swapneel Deshpande is a solution architect, development consultant, and a trusted technology advisor, entrepreneur, start-up consultant, and more. He has been working in IT for nearly two decades, with expertise in .NET and web-related technologies. He has led the software architecture design, development, and delivery of large, complex solutions. He is passionate about embracing new technologies and teaching. His current role involves leading large and complex projects right from architecture to delivery.
To my family for their unwavering support, patience, and encouragement throughout the process of writing this book. I am also grateful for my elder sister who has always been a source of motivation for me.
Henry Been has been working in IT for over ten years. He is an independent architect, developer, and trainer, who has worked with many different companies. With many of these companies, Henry has embarked on a journey implementing practices such as continuous integration and deployment, infrastructure as code, trunk-based development, and implementing feedback loops.
Alongside his work, Henry creates online training courses for Pluralsight and A Cloud Guru, and frequently speaks at meetups and conferences. He has held the Microsoft MVP award since 2019.
Maik van der Gaag is the CTO at 3fifty, an experienced consultancy company with a strong focus on the Microsoft cloud. He has over 15 years of experience providing architecture, development, training, and design expertise. During his career, Maik has worked on a variety of projects, ranging from cloud transformations to DevOps implementations.
Maik loves to share his knowledge, which was also one of the reasons why he founded the Dutch Cloud meetup. Maik is a Microsoft MVP, public speaker, talented blog writer, and an organizer of events.
Shachaf Goldstein has 10 years of experience in the IT infrastructure and development fields alongside 3 years of experience in cloud infrastructure and solutions. He practices various DevOps methodologies and works with many platforms (GitHub, Azure DevOps, Atlassian Jira, GitLab). Shachaf is an experienced professional in automated cloud deployments, both for code and infrastructure elements, with a focus on Azure in recent years.
Jamel Achahbar has over 10 years of experience training people in several domains, is especially passionate about conveying technical knowledge to his audience in a practical way, and is an Azure geek. He has delivered Azure Immersion Workshops, created practical labs and blog content, and is a technical lead at his current company. He enjoys supporting his colleagues to achieve success for their customers. He has been instrumental in the move to the cloud of several customers. He has a degree in network and systems administration and is a Certified Azure Solutions Architect Expert and Microsoft Certified Trainer. He likes to get out of his comfort zone and take on challenges to keep learning and growing. Jamel has a never-ending learning attitude.
I’d like to thank my family and friends who understand the time and commitment it takes to keep abreast of everything that happens in the cloud. Working in this field would not be possible without the support system of a great Microsoft cloud community, passionate leaders, and great friends and family. Thank you to all of the giants who created these great tools, content, and services to make our work easier and help us achieve success!
Every software engineering professional must be adept in their understanding of DevOps and the positive impact it brings to their teams. This book acquaints you with some of the most important DevOps practices, such as configuration management, release management, continuous integration, continuous deployment, infrastructure as code, continuous testing, and application monitoring and feedback. You will also learn to use tools such as Azure DevOps, GitHub, and other related tools for the implementation of the various practices.
With detailed case studies and hands-on labs, this book serves as a ready reckoner on how to implement the various practices. This book also covers additional topics that will help you develop deep expertise and a DevOps-oriented mindset, which will be impactful in your professional career.
The target audience for this book is the following:
Software developers or operations specialists who want to undertake the AZ-400 exam Software developers or operations specialists who want to learn how to use Azure DevOps to implement DevOps practicesTechnology enthusiasts who would like to enhance their DevOps skillsYou should be familiar with software development and, more broadly, software development practices. You must be familiar with terms such as Waterfall, Scrum, Agile, and DevOps. Any serious practitioner with at least 2 years of team-based software development experience should be able to read this book. You should also have a basic understanding of Azure DevOps and GitHub.
Chapter 1, Introduction to DevOps, provides an overview of the DevOps movement and the culture it represents. This chapter introduces the practices and habits that will be the focus of this book.
Chapter 2, Site Reliability Engineering Fundamentals, introduces you to the principles and practices of Site Reliability Engineering (SRE), as well as the significance of adopting an SRE mindset to ensure the success of cloud applications.
Chapter 3, Getting the Best Out of DevOps Tools, provides a high-level overview of the various DevOps-related tools that can be used throughout an application development life cycle.
Chapter 4, Everything Starts with Source Control, discusses various approaches to source control and how proper source control is at the heart of continuous delivery.
Chapter 5, Moving to Continuous Integration, covers continuous integration, automated testing, quality control, metrics, code coverage, and the minimum standards necessary to produce a quality product.
Chapter 6, Implementing Continuous Deployment and Release Management, covers how to use the Azure DevOps release pipeline, as well as various deployment patterns and strategies to enable continuous deployment.
Chapter 7, Dependency Management, looks at how Azure Artifacts can help to manage shared components and how to use Azure pipelines to automate the build, release, and even use of these components.
Chapter 8, Implement Infrastructure and Configuration as Code, shows how to configure your application’s infrastructure and how to fully automate the creation or updating of that infrastructure as part of your release.
Chapter 9, Dealing with Databases in DevOps Scenarios, discusses several strategies for treating your database like application code, an infrastructure, or a configuration.
Chapter 10, Integrating Continuous Testing, looks at different types of testing and how they relate to one another using the testing pyramid and how different types of tests can be embedded at different stages of our DevOps pipelines.
Chapter 11, Managing Security and Compliance, shows you how to collaborate with security experts to automate their concerns in a pipeline, rather than manually verifying them later using checklists.
Chapter 12, Application Monitoring, explores application monitoring, which is the process of instrumenting your application to understand application performance and usage patterns and extract key performance indicators.
Chapter 13, Gathering User Feedback, discusses how to use hypothesis-driven development to validate your ideas before putting them into action.
Chapter 14, Adopting the Culture of Continuous Improvement, concentrates on some aspects of organizational culture that have a direct influence on business outcomes.
Chapter 15, Accelerate Cloud Adoption through DevOps, provides an overview of using DevOps through scenarios and guidance while preparing an enterprise cloud adoption plan.
Chapter 16, Containers, provides an overview of containers, which allow you to package any application or tool (developed using any programming language) and deploy it on a basic host or cluster.
Chapter 17, Planning Your Azure DevOps Organization, discusses what works and what doesn’t for your organization and team, as well as how to implement practices and approaches using Azure DevOps.
Chapter 18, AZ-400 Mock Exam, contains a mock test for you to assess your learning.
Software/hardware covered in the book
Operating system requirements
Visual Studio 2019/Visual Studio 2022/VS Code
Windows, macOS, or Linux
An Azure subscription
-
An Azure DevOps subscription
-
If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Designing-and-Implementing-Microsoft-DevOps-Solutions-AZ-400-Exam-Guide. If there’s an update to the code, it will be updated in the GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/OADjU.
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: In GitHub Flow, there is one master branch that should always be in a deployable state. No unfinished changes are allowed to go onto the branch.
A block of code is set as follows:
{ "appServiceName": { "type": "string", "metadata": { "description": "a free to choose text" } }When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
az aks update -n 'packtsbookaci' -g 'az400-dev' --attach-acr 'packtbookacr'Any command-line input or output is written as follows:
git add NewFile.txt
git push
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Select System info from the Administration panel.”
Tips or Important Notes
Appear like this.
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Once you’ve read Designing and Implementing Microsoft DevOps Solutions AZ-400 Exam Guide, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.
In this part, you’ll learn about the core principles and practices of DevOps that are foundational to achieving your software development goals in a faster and more efficient manner. We will explore how most organizations can build secure and reliable systems by incorporating Site Reliability Engineering (SRE) practices. You will also learn about the important DevOps tools that can improve the productivity of your teams.
This section establishes the necessary foundations for our work, after which the remaining chapters can be read (in any order) to get deeper insights into the respective topics.
This part of the book comprises the following chapters:
Chapter 1, Introduction to DevOpsChapter 2, Site Reliability EngineeringFundamentalsChapter 3, Getting the Best Out of DevOps ToolsDevOps is not a product or tool that you can buy or install. DevOps is about culture and the way you write, release, and operate your software. DevOps is about shortening the time between a new idea and your first end user experiencing the value it delivers. In this book, you will learn about the tools and techniques you can use to apply that philosophy to your way of working. As the purpose of this book is to help you prepare for the AZ-400 certification exam, the core concepts and DevOps-related practices will be illustrated using Azure DevOps and its associated technologies.
DevOps has gained popularity in recent years, with almost all software engineering teams adapting themselves to a new world of doing things using a newer set of tools that activate productivity and better collaboration across development and operations. DevOps is often seen as an extension of Agile, but its scope is much wider, wherein it complements Agile.
In this first chapter, you will learn more about what DevOps is and how to recognize a successful DevOps team. By the end of this chapter, you will be familiar with the key terms and be equipped with a broader understanding of the high-level principles and practices of DevOps. You will also discover the benefits of DevOps as it applies to software engineering teams.
In this chapter, we will cover the following topics:
What is DevOps?Creating your ideal DevOps organizationExploring DevOps practices and habitsThe five stages of the DevOps evolutionThere are no technical requirements for this chapter.
If you were to list all of the different definitions and descriptions of DevOps, there would be many. However, as different as these might be, they most likely share several concepts. These are collaboration, continuous delivery of business value, and breaking down silos.
With all the technical discussion in the rest of this book, it is important not to overlook the value proposition for adopting DevOps – namely, that it will help you improve the way that you continuously deliver value to your end users. To do this, you must decrease the time between starting work on a new feature and the first user using it in production. This means that you not only have to write the software but also deliver and operate it.
Over the last decade, the way we write software has fundamentally changed. More and more companies are now adopting an agile way of working to increase the efficiency of their software development. More and more teams are now working in short iterations or sprints to create new increments of a product in quick succession. However, creating potentially shippable increments faster and faster does not create any value by itself. Only when each new version of your software is also released to production and used by your end users does it start delivering value.
In traditional organizations, developers and operators are often located in different departments, and taking software into production includes a hand-off, often with a formal ceremony around it. In such an organization, it can be hard to accelerate that delivery to production, along with the speed at which development can create new versions.
Next to that, the development and operations departments often have conflicting goals. While a development department is rewarded for creating many changes as fast as possible, operations departments are rewarded for limiting downtime and preventing issues. The latter is often best achieved by making as few changes as possible. The conflict here is clear – both departments have optimizations for one subgoal, as shown in the following diagram:
Figure 1.1 – Conflicting goals between development and operations
This defeats the purpose of these subgoals, which comes from the shared, overarching goal of quickly taking in new versions while maintaining stability. It’s precisely this conflict between developmental and operational goals that is one of the things that should disappear in a DevOps culture. In such a culture, development and operations teams should work together on delivering new versions to production in a fast and reliable manner and share responsibility for both subgoals.
While it is good to know that DevOps is a cultural movement, tools and automation are an important part of that culture. In this book, we will focus on these tools and how to use them to implement many of the practices that come with a DevOps culture. In other words, this book will be mostly about the products and processes associated with DevOps. If you want to learn more about the cultural side of things and the people, there are many other books you can read. A very good read is The Phoenix Project: A Novel About IT, DevOps, And Helping Your Business Win, by Gene Kim.
The rest of this section will explore the relationship between DevOps and Agile to see how they complement each other. The focus will be on agile techniques and prices for work management. We will also discuss the goals and benefits of a DevOps culture.
If you take a look at Agile, you may notice that part of it focuses on business value and shortening the time of delivering a new business value. From that perspective, adopting DevOps is a logical next step after Agile. Agile advocates that the software development teams’ responsibilities should extend forward by engaging with users and other stakeholders to deliver valuable and potentially shippable products quicker. DevOps is not just about something that might be shipped, but shipping it as well. With Agile and DevOps combined, you can create an end-to-end and continuous flow of value to your users.
You will need a common approach to managing the work to be done for everyone involved. In the next section, you will find some pointers on how to incorporate operational concerns in the way you manage your work.
When you are starting to increase the collaboration between development and operations, you will quickly notice that they have to cope with different types of work. In development, a large part of the work is planned: user stories and bugs that are picked up from a backlog. On the other hand, for operations, a large part of their work is unplanned. They respond to warnings and alerts from systems and requests or tickets from users or developers.
Integrating these two, especially if developers and operators are located on the same team, can be challenging. To learn how to deal with this, let’s explore the following approach:
First, switch to a flow-based way of working for developers.Next, allow for operations to also list their work in the same work management system as developers using synchronizations. You can also choose to implement fastlaning, a way to expedite urgent work.Finally, you may choose to decommission existing ticketing tools for operations if possible.Fastlaning is an approach to organizing work that allows for both planned and unplanned work by visualizing two separate lanes of work. To do this, the Scrum board is extended with a Kanban-like board on the top. This is the fast lane. On the Kanban board, urgent but unplanned work is added. Any work that’s added to this lane is picked up by the team with the highest priority. Only when there is no work remaining in the fast lane is work from the Scrum board, along with planned work, picked up. Whenever new work is added to the fast lane, this takes priority again. Often, there is the agreement that any work in progress is finished before switching to work in the fast lane.
Important Note
Dependency management is also an important aspect of agile work planning. Hence, teams often make use of the prioritization attribute to qualify more important work for the short term.
The first thing to consider when switching to a flow-based methodology is transitioning the way developers work from batch-wise to flow-based. An example of a batch-wise way of working is Scrum. If you are using the Scrum framework, you are used to picking up a batch of work every 2 to 4 weeks and focusing on completing all of that work within that time window. Only when that batch is done do you deliver a potentially shippable product.
When changing to a flow-based approach, you try to focus not on a batch, but just on one thing. You work on that one work item and drive it completely until it’s done before you start on the next. This way, there is no longer a sprint backlog, only a product backlog. The advantage of this approach is that you no longer decide which work to perform upfront; whenever you are free to start on new work, you can pick up the next item from the backlog. In an environment where priorities shift quickly, this allows you to react to change quickly.
These changes to the way developers organize their work make it easier to include operations in work management, but there is also another benefit. When developers are focusing on getting a single work item done instead of a whole sprint at once, you can also increase the number of times you can deliver a small portion of value to your users.
Once the development team has changed the way it organizes its work, it should be easier for developers to also list their planned work on the shared backlog and pull work from that backlog when they have time to work on it. They now also have a place where they can list their unplanned work.
However, there may still be an existing ticketing system where requests for operations are dropped by users or automatically created by monitoring tools. While Azure DevOps has a great API to rework this integration to directly create work items in Azure DevOps, you may first choose to create a synchronization between your existing ticketing tool and Azure Boards. There are many integration options available and there is a lot of ongoing work in this area. This way, operators can slowly move from their old tool to the new one, since they are now in sync. Of course, the goal is for them to move over to the same tool, as the developers, completely.
With the work of developers and operators in the same work management tool, you will notice that you have a mix of planned and unplanned, often urgent, work in the system. To ensure that urgent work gets the attention and precedence it deserves, you can introduce what is called a fast lane to your sprint board. The following screenshot shows an example of an Azure board that has been set up for fastlaning production issues:
Figure 1.2 – Azure Board setup depicting the fast lane
The horizontal split in this board is only used to work on tasks in the regular lane when there is no work to be picked up in the fast lane.
You can find instructions on how to configure swim lanes in your Azure (Kanban) boards for expediting work at https://docs.microsoft.com/en-us/azure/devops/boards/boards/expedite-work?view=azure-devops.
After creating a shared work management system between development and operations, there is an opportunity to increase the amount of collaboration between them. When this collaboration is taking off, old ticketing systems that were used by operations may now slowly be decommissioned over time. Integrations from monitoring tools can be shifted to the new shared tools, and the number of tickets between developers and operators should slowly decrease as they find new ways of working together.
Important Note
Azure DevOps allows you to customize work item templates, as well as define life cycle states. Using this feature, teams can easily model their work item template types based on any existing taxonomy they might be using in their existing tools. This significantly reduces the learning curve in the adoption of the new shared work management tool. For more information on this, go to https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/work-item-template?view=azure-devops&tabs=browser#manage-work-item-templates.
At this point, you might be wondering about the point of it all. What are the benefits of DevOps and what’s in it for you, your colleagues, and your organization? The most common goal of adopting DevOps is to achieve a reduction in cycle time. Cycle time is the time between starting work on a new feature and the moment that the first user can use it. The way this is achieved, by automation, also serves the goals of lower change failure rate, lower mean time to repair (MTTR), and lower planned downtime.
Next to all that, there may be other benefits, such as increased employee satisfaction, less burnout and stress, and better employee retention. This is attributed to removing opposing goals between developers and operators.
For a while, there was doubt about whether DevOps works, whether these goals can be met, and whether the extra benefits can be achieved since this was only shown using case studies. The downside of this is that case studies are often only available for successful cases, not for unsuccessful ones. This all changed in 2018 when the book Accelerate came out. This book shows, based on years of quantitative research, that modern development practices such as DevOps contribute to reaching IT goals and organizational goals.
To measure where you currently stand as a team or organization and the impact DevOps has on you, there are several metrics that you can start recording. As always, when working with metrics or key performance indicators (KPIs), make sure that you do not encourage people to game the system by looking only at the numbers. Several interesting metrics are detailed in the following sections and if you go over them, you will notice that they are all about encouraging flow.
Cycle time and lead time are metrics that come from Lean and Kanban and are used to measure the time needed to realize a change. Cycle time is the amount of time between starting work on a feature and users being able to use that feature in production. The lower the cycle time, the quicker you can react to changing requirements or insights. Lead time is the amount of time between requesting a feature and realizing that feature. It is the time between adding work to the backlog and when you start implementing it.
When you add cycle time and lead time together, you are calculating another metric, known as the time to market. This is often an important business metric when developing software. Hence, minimizing both cycle time and lead time will have a business impact.
Another thing you can measure is the amount of work in progress at any point in time. DevOps focuses on the flow of value to the user. This implies that everyone should, if possible, be doing only one thing at a time and finish that before moving on to something else. This reduces the amount of time spent on task switching and the amount of time spent on not yet complete work. Measuring how many things a team works on in parallel and reporting on this can act as a source of encouragement.
You can even go as far as putting actual limits on the amount of work that can be in progress. The following is a small part of Figure 1.2, showing that these work-in-progress limits can even be shown in the tool:
Figure 1.3 – Azure Boards depicting limits for each stage
The goal is to have as little work in progress at the same time as possible.
The third metric is the mean time to recovery. How long does it take you to restore a service in case of a (partial) outage? In the past, companies focused on reducing the mean time between failures. This used to be the mean indicator of the stability of a product. However, this metric encourages limiting the number of changes going to production. The unwanted consequence is often that outages, though they might be rare, last long and are hard to fix.
Measuring the mean time to recovery shifts the attention to how quickly you can remediate an outage. If you can fix outages quickly, you can achieve the same – namely, you can minimize the amount of downtime without sacrificing the rate of change. The goal is to minimize the time to recovery.
Finally, you can measure the number of changes that are delivered to production and the percentage of that which is not successful. Increasing the rate of change implies that you are delivering value to your users more often, hence realizing a flow of value. Also, by measuring not just the number of failures but also the percentage that fails, you are encouraging many small, successful changes instead of encouraging whether the number of changes is limited overall.
Your goal should be to increase the rate of change while lowering the change failure rate. Apart from the four major KPIs listed in this section, many other metrics may be useful in measuring your DevOps maturity. All these metrics must be linked back to the important business objectives and key results (OKRs) that are expected. You can find more information about OKRs here: https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/strategy/business-outcomes/okr.
A representative sample, for illustration purposes, is depicted in the following table:
Objective
Key Results
Faster time to market
Deployment Frequency: Every weekDeployment Time <= 4 hoursLead Time (Major Releases): Once every quarterIncrease the business value that’s been realized while maintaining or reducing costs
CI/CD processes: 100% automatedResource Utilization (95th percentile): 80%Dashboards for monitoring both Health and CostsPredictable and quality delivery and faster correction with fewer defects
High Availability > 99.9%RTO < 1 hour, RPO < 15 minsBetter processes across IT, automation, teamwork, and culture
MTTR < 1 hourLead Time (Bugs) < 8 hourScaled Agile: Feature Teams > 5 Technical Debt < 1 weekImproved customer engagement and ability to quickly respond to market demands
CSAT: 4 or aboveProduct Planning: 50% of the backlog focuses on Customer FeedbackTable 1.1 – Using the OKR approach for your DevOps maturity
At this point, you might be wondering, how do I help my organization foster this culture and reap all of these benefits? The next section will answer this.
Well, maybe your organizational structure does not have to change at all. DevOps must start with a cultural change: openness, empathy, and collaboration are values that need to be encouraged. But still, changing your organizational structure may help accelerate this.
Traditionally, developers and operators are often organized into disparate teams or even different departments – organized in teams with people that have similar skill sets and responsibilities. A common change in organizations is changing this structure by pivoting and organizing teams behind a common goal, a single product, or a group of features, for example.
Now, you will need teams with different skill sets and responsibilities, teams most likely with developers and operators. It is important to realize that forcing such a change upon these people may not be the best way forward. Often, it works best to start with changing the culture and encouraging cooperation – then, this organizational change may come about naturally.
Finally, it is important to recognize one anti-pattern at this point. Some companies are trying to implement DevOps by hiring specialized DevOps engineers and positioning them between development and operations, interacting with both. While this, at first, may seem like a good idea, this goes against the DevOps values. If you do this, you are not breaking silos down, but you are adding a third one. You are not decreasing the number of hand-offs, you are most likely increasing them. Also, collaboration between developers and operations is often not enhanced by separating them using another organizational structure, and you may not see any increase in value to your end users at all.
Now that you know what DevOps is and you have a clear understanding of how you can form a DevOps team, it is time to explore how to start achieving your goals.
Since you are not the first team going on this journey, you can learn from the experiences of those before you. One example is the Microsoft team that built Azure DevOps. Being in the rare position that they can use their product for developing another product, they have learned a great deal about what makes DevOps successful. From this, they have identified seven key DevOps practices and seven DevOps habits that many successful DevOps teams share:
DevOps Practices
DevOps Habits
Configuration management
Team autonomy and enterprise alignment
Release management
Rigorous management of technical debt
Continuous integration
Focus on the flow of customer value
Continuous deployment
Hypothesis-driven development
Infrastructure as Code
Evidence gathered in production
Test automation
Live-site culture
Application performance monitoring
Myanage infrastructure as a flexible resource
Table 1.2 – DevOps practices and habits
Now, it is important to realize that just copying the motions described here will not guarantee success. Just as with Agile, you will have to spend time to understand these practices and habits, where they come from, and how they contribute to a continuous flow of value to your end users.
Important Note
Microsoft Services has introduced a more comprehensive DevOps Dojo model that aims to categorize the maturity of your DevOps practices. It gives you a good structure on how you should prioritize the DevOps-related investments within your teams. You can read more about it here: https://docs.microsoft.com/en-us/learn/paths/devops-dojo-white-belt-foundation/.
The following sections explore all of these practices and habits in more detail. Keep these in the back of your mind while reading the rest of this book. While the rest of this book will mostly focus on the technical means of how to do things, do not forget that these are only means. The real value comes from your mindset and creating a culture that focuses on creating a continuous flow of value for your customers.
This section discusses all seven DevOps practices in turn. As you will quickly see, they are highly related, and it is quite hard to practice one without the other. For example, test automation is highly related to continuous integration and continuous deployment.
Important Note
If you plan on taking the AZ-400 exam, mastering all of these practices and performing them using Azure DevOps will help you significantly.
Configuration management is about versioning the configuration of your application and the components it relies on, along with your application itself. Configuration is kept in source control and takes the form of, for example, JSON or YAML files that describe the desired configuration of your application. These files are the input for tools such as Ansible, Terraform, Puppet, or PowerShell DSC, which configure your environment and application. These tools are often invoked from a continuous deployment pipeline.
The desired state can also be reapplied at an interval, even if no changes have been made to the intended configuration. This way, it is ensured that the actual configuration stays correct and that manual changes are automatically revoked. We call this the prevention of configuration drift. Configuration drift occurs over time due to servers being added or removed over time, or manual, ad hoc interventions by administrators. Of course, this implies that intended updates to the configuration are done in source control and only applied using tools.
Configuration management or Configuration as Code (CaC) is highly related to Infrastructure as Code (IaC). The two are often intertwined and on some platforms, the difference between the two may even feel artificial. CaC will be discussed in detail in Chapter 8, Implement Infrastructure and Configuration as Code.
Release management is about being in control of which version of your software is deployed to which environment. Versions are often created using continuous integration and delivery pipelines. These versions, along with all of the configuration needed, are then stored as immutable artifacts in a repository. From here on, release management tools are used to plan and control how these versions are deployed to one or more environments. Examples of such controls include manual approvals and automated queries of open work and quality checks before allowing deployment to a new environment.
Release management is related to continuous deployment and focuses more on controlling the flow of versions through the continuous deployment pipeline. Chapter 8, Implement Infrastructure and Configuration as Code, will cover CaC as part of release management.
Continuous integration is a practice where every developer integrates their work with that of the other developers in the team at least once a day and preferably more often. This means that every developer should push their work to the repository at least once a day. A continuous integration build verifies that their work compiles and that all the unit tests run. It is important to understand that this verification should not run only on the code that the developer is working on in isolation. The real value comes when the work is also integrated with the work of others.
When integrating changes often and fast, problems with merging changes are less frequent and if they occur, they are often less difficult to solve. In Chapter 4, Everything Starts with Source Control, you will learn more about how to set up your source control repositories to make this possible. In Chapter 5, Moving to Continuous Integration, you will learn about setting up a continuous integration build.
Continuous deployment is the practice of automatically deploying every new version of sufficient quality to production. When practicing continuous deployment, you have a fully automated pipeline that takes in every new version of your application (every commit), results in a new release, and starts deploying it to one or more environments. The first environment is often called test and the final environment is called production.
In this pipeline, multiple steps verify the quality of the software before letting it proceed to the next environment. If the quality is not sufficient, the release is aborted and will not propagate to the next environment. The premise behind this approach is that, in the pipeline, you try to prove that you cannot take the current version to the next environment. If you fail to prove so, you assume it is ready for further progression.
Only when a release has gone through all the environments in the pipeline is it deployed to production. Whenever a release cannot progress to the next environment, that release will be completely canceled. While you might be inclined to fix the reason for the failure and then restart deployment from the point where it failed, it is important not to do so. The changes you made at that point are not validated by all of the controls that the version has already passed through. The only way to validate the new version as a whole is by starting the pipeline from the start. You can see this clearly in the following diagram:
Figure 1.4 – Continuous deployment flow
In Chapter 6, Implementing Continuous Deployment and Release Management, you will learn about setting up continuous deployment using.
Important Note
The preceding diagram can be found at https://en.wikipedia.org/wiki/Continuous_delivery#/media/File:Continuous_Delivery_process_diagram.svg. The image is by Grégoire Détrez, original by Jez Humble, under CC BY-SA 4.0, at https://creativecommons.org/licenses/by-sa/4.0/.
When writing an application, the binaries that you are building must be running somewhere on some application host. An example of such an application host can be a web server such as IIS or Apache. Next to an application host, we may need a database and some messaging solution. This is called the infrastructure for our application. When practicing IaC, you are keeping a description of this infrastructure in your source code repository, alongside your application code.
When the time comes to release a new version of the application and you need to make one or more changes to the infrastructure, you are executing this description of your desired infrastructure using tools such as Chef, Puppet, Terraform, Azure Bicep, PowerShell DSC, or Azure ARM templates. The execution of such a description is idempotent, which means that it can be executed more than once, and the result is the same. This is because your description of the infrastructure describes the desired state you want the infrastructure to be in and not a series of steps to be executed. Those steps to be executed, if there are any, are automatically determined by your tool of choice. Applying the desired state can also be done automatically in a continuous deployment pipeline and is often executed before the application code is updated.
The big advantage of this is that you can easily create a new environment where the infrastructure is guaranteed to be the same as in your other environments. Also, the problem of configuration drift, where the infrastructure between your different environments slowly diverges, is no longer possible since every time you apply the desired state again to every environment, it’s forced.
Chapter 8, Implement Infrastructure and Configuration as Code, will discuss IaC in more detail.
To continuously deliver value to your end users, you must release fast and often. This has implications for the way you test your application. You can no longer execute manual tests when you release your application every few minutes. This means that you must automate as many of your tests as possible.
You will most likely want to create multiple test suites for the applications that you run at different stages of your delivery pipeline. Fast unit tests that run within a few minutes and that are executed whenever a new pull request is opened should give your team very quick feedback on the quality of their work, and it should catch most of the errors. Next, the team should run one or more slower test suites later in the pipeline to further increase your confidence in the quality of a version of your application.
All of this should limit the amount of manual testing to a bare minimum and allow you to automatically deploy new versions of your application with confidence.
Chapter 10, Integrating Continuous Testing, will cover test automation in detail.
This last practice is all about learning how your application is doing in production. Gathering metrics such as response times and the number of requests will tell you about how the systems are performing. Capturing errors is also part of performance monitoring and allows you to start fixing problems without having to wait on your customers to contact you about them.
In addition to that, you can gather information about which parts of the application are more or less frequently used and whether new features are being picked up by users. Learning about usage patterns provides you with great insights into how customers use your applications and common scenarios they are going through.
Chapter 11, Managing Security and Compliance, and Chapter 12, Application Monitoring, will go into detail about both your application and your users’ behavior in production.
The seven habits of successful DevOps teams are more concerned with culture and your attitude while developing and delivering software, and less with technical means than DevOps practices are. Still, it is important to know and understand these habits since they will help make DevOps adoption easier.
You will notice that developing these habits will reinforce the use of the practices enumerated previously and the tools you use to implement them. And of course, this holds the other way around as well.
An important part of working Agile is creating teams that are largely self-directed and can make decisions without (too many) dependencies outside the team. Such a team will often include multiple roles, including a product owner that owns one or more features and is empowered to decide on their way forward.
However, this autonomy also comes with the responsibility to align the work of the team with the direction the whole product is taking. It is important to develop ways of aligning the work of tens or hundreds of teams with each other, in such a way that everyone can sail their course but the fleet as a whole stays together as well.
The best-case scenario is that teams take it upon themselves to align to the larger vision, instead of taking directions now and then.
Another habit is that of rigorous management of technical debt. The term debt in itself suggests that there is a cost (interest) associated with the delay in addressing an issue. To keep moving at a constant pace and not slowly lose speed over time, it is crucial to keep the number of bugs or architectural issues to a minimum and only tolerate so much. Within some teams, this is even formalized in agreements. For example, a team can agree that the number of unfixed bugs should never exceed the number of team members. This means that if a team has four members and a fifth bug is reported, no new work will be undertaken until at least one bug has been fixed.
It is important to accept that users receive no value from code that has been written until they are using it. Focusing on the flow of value to a user means that code must be written, tested, and delivered and should be running in production before you are done. Focusing on this habit can drive cooperation between disciplines and teams.
In many modern development methodologies, there is a product owner who is responsible for ordering all of the work in the backlog, based on the business value. This owner, as the expert, is responsible for maximizing the value that’s delivered by the development team by ordering all the items based on the business value (divided by effort).
However, recent research has shown that even though the product owner is an expert, they cannot correctly predict which features will bring the most value to users. Roughly one-third of the work from a team adds value for users, while another third decreases the value. For this reason, you can switch your backlog from features or user stories to the hypothesis you want to prove or disprove. You only create a minimal implementation or even just a hint of a feature in the product and then measure whether it is picked up by users. Only when this happens can you expand the implementation of the feature.
Performance measurements should be taken in your production environment, not (just) in an artificial load test environment. There is nothing wrong with executing load tests before going to production if they deliver value to you. However, the real performance effort is done in the production environment. It should be measured there and compared with previous measurements.
This also holds for usage statistics, patterns, and many, many other performance indicators. They can all be automatically gathered using production metrics.
A live-site culture promotes the idea that anything that happens in the production environment takes precedence over anything else. Next, anything that threatens production, is about to go to production, or hinders going to production at any time gets priority. Only when these are all in order is the attention shifted to future work.
Also, a part of a live-site culture is ensuring that anything that disturbed the operation of the service is thoroughly analyzed – not to find out who to blame or fire but to find out how to prevent this from happening again. Prevention is preferably done by shifting left; for example, by detecting an indicator of a repeat incident earlier in the pipeline.
Finally, a successful DevOps team treats its servers and infrastructure as mutable assets that can be continually changed, upgraded, tuned, or even decommissioned to meet the demands of the business. The ability to do this is fueled by configuration and IaC. This may even go so far as creating a new production environment for every new deployment and just deleting the old production environment after switching all traffic from the old environment to the new one.
Besides keeping these DevOps practices and habits in mind, there are certain stages that you will go through while trying to move to a DevOps culture in your organization. The next section will take you through them.
When you are trying to move to a DevOps culture in your organization, it is going to take time. There are motions you have to go through while everyone in your organization embraces the changes they have to make to their ways of working. Others that have gone before you have gone through the following five steps or stages, which may help you. Knowing about them can help you accelerate your journey. These steps were first published in the 2018 State of DevOps Report and are discussed in the following sections.
A common
