70,79 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Learn to use some of the most exciting and powerful tools to deliver world-class quality software with continuous delivery and DevOps
About This Book
- Get to know the background of DevOps so you understand the collaboration between different aspects of an IT organization and a software developer
- Deploy top-quality software and ensure software maintenance and release management with this practical guide
- This course covers some of the most exciting technology available to DevOps engineers, and demonstrates multiple techniques for using them
- Real-world and realistic examples are provided to help you as you go about the implementation and adoption of continuous delivery and DevOps
Who This Book Is For
This course is for developers who want to understand how the infrastructure that builds today's enterprises works, and how to painlessly and regularly ship quality software.
What You Will Learn
- Set up and familiarize yourself with all the tools you need to be efficient with DevOps
- Design an application that is suitable for continuous deployment systems with DevOps in mind
- Test the code using automated regression testing with Jenkins Selenium
- Managing the lifecycle of hosts, from creation to ongoing management using Puppet Razor
- Find out how to manage, use, and work with Code in the Git version management system
- See what traps, pitfalls, and hurdles to look out for as you implement continuous delivery and DevOps
In Detail
Harness the power of DevOps to boost your skill set and make your IT organization perform better. If you're keen to employ DevOps techniques to better your software development, this course contains all you need to overcome the day-to-day complications of managing complex infrastructures the DevOps way.
Start with your first module – Practical DevOps - that encompasses the entire flow from code from testing to production. Get a solid ground-level knowledge of how to monitor code for any anomalies, perform code testing, and make sure the code is running smoothly through a series of real-world exercise, and develop practical skills by creating a sample enterprise Java application.
In the second module, run through a series of tailored mini-tutorials designed to give you a complete understanding of every DevOps automation technique. Create real change in the way you deliver your projects by utilizing some of the most commendable software available today. Go from your first steps of managing code in Git to configuration management in Puppet, monitoring using Sensu, and more.
In the final module, get to grips with the continuous delivery techniques that will help you reduce the time and effort that goes into the delivery and support of software.
This Learning Path combines some of the best that Packt has to offer in one complete, curated package. It includes content from the following Packt products:
- Practical DevOps by Joakim Verona
- DevOps Automation Cookbook by Michael Duffy
- Continuous Delivery and DevOps : A Quickstart Guide - Second Edition by Paul Swartout
Style and approach
This course is an easy to follow project based guide for all those with a keen interest in deploying world-class software using some of the most effective and remarkable technologies available.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 894
Veröffentlichungsjahr: 2016
Ähnliche
Table of Contents
Learning DevOps: Continuously Deliver Better Software
Learning DevOps: Continuously Deliver Better Software
Learn to use some of the most exciting and powerful tools to deliver world-class quality software with continuous delivery and DevOps
A course in three modules
BIRMINGHAM - MUMBAI
Learning DevOps: Continuously Deliver Better Software
Copyright © 2016 Packt Publishing
All rights reserved. No part of this course may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this course to ensure the accuracy of the information presented. However, the information contained in this course is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this course.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this course by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Published on: September 2016
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78712-661-9
www.packtpub.com
Credits
Authors
Joakim Verona
Michael Duffy
Paul Swartout
Reviewers
Per Hedman
Max Manders
Jon Auman
Tom Geudens
Sami Rönkä
Diego Woitasen
Adam Strawson
Content Development Editor
Aishwarya Pandere
Production Coordinator
Nilesh Mohite
Preface
The Learning DevOps: Continuously Deliver Better Software, is a course that will harness the power of DevOps to boost your skill set and make your IT organization perform better. It will aid developers, systems administrators who are keen to employ DevOps techniques to help with the day-to-day complications of managing complex infrastructures.
What this learning path covers
Module 1, Practical DevOps, describes how DevOps can assist us in the emerging field of the Internet of Things.
Module 2, DevOps Automation Cookbook, covers recipes that allow you to automate the build and configuration of the most basic building block in your infrastructure servers..
Module 3, Continuous Delivery and DevOps – A Quickstart Guide - Second Edition, provides some insight into how you can take CD and DevOps techniques and experience beyond the traditional software delivery process.
What you need for this learning path
Module 1: This module contains many practical examples. To work through the examples, you need a machine preferably with a GNU/Linux-based operating system,
such as Fedora.
Module 2: For this book, you will require the following software:
A server running Ubuntu 14.04 or greater.
A desktop PC running a modern Web Browser
A good Text editor or IDE.
Module 3: There are many tools mentioned within the book that will help you no end. These include technical tools such as Jenkins, GIT, Docker, Vagrant, IRC, Sonar, and Graphite, and nontechnical tools and techniques such as Scrum, Kanban, agile, and TDD.
You might have some of these (or similar) tools in place, or you might be looking at implementing them, which will help. However, the only thing you’ll really need to enjoy and appreciate this book is the ability to read and an open mind.
Who this learning path is for
This course is for developers who wish to take on larger responsibilities and understand how the infrastructure that builds today’s enterprises works. It is also for operations personnel who would like to better support their developers. Anyone who wants to understand how to painlessly and regularly ship quality software can take up this course.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this course—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the course’s title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt course, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this course from your account at http://www.packtpub.com. If you purchased this course elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the course’s webpage at the Packt Publishing website. This page can be accessed by entering the course’s name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
The code bundle for the course is also hosted on GitHub at https://github.com/PacktPublishing/Learning-DevOps-Continuously-Deliver-Better-Software. We also have other code bundles from our rich catalog of books, videos, and courses available at https://github.com/PacktPublishing/. Check them out!
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our courses—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this course. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your course, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the course in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this course, you can contact us at <[email protected]>, and we will do our best to address the problem.
Part I. Module 1
Practical DevOps
Harness the power of DevOps to boost your skill set and make your IT organization perform better
Chapter 1. Introduction to DevOps and Continuous Delivery
Welcome to Practical DevOps!
The first chapter of this book will deal with the background of DevOps and setting the scene for how DevOps fits into the wider world of Agile systems development.
An important part of DevOps is being able to explain to coworkers in your organization what DevOps is and what it isn't.
The faster you can get everyone aboard the DevOps train, the faster you can get to the part where you perform the actual technical implementation!
In this chapter, we will cover the following topics:
Introducing DevOps
DevOps is, by definition, a field that spans several disciplines. It is a field that is very practical and hands-on, but at the same time, you must understand both the technical background and the nontechnical cultural aspects. This book covers both the practical and soft skills required for a best-of-breed DevOps implementation in your organization.
The word "DevOps" is a combination of the words "development" and "operation". This wordplay already serves to give us a hint of the basic nature of the idea behind DevOps. It is a practice where collaboration between different disciplines of software development is encouraged.
The origin of the word DevOps and the early days of the DevOps movement can be tracked rather precisely: Patrick Debois is a software developer and consultant with experience in many fields within IT. He was frustrated with the divide between developers and operations personnel. He tried getting people interested in the problem at conferences, but there wasn't much interest initially.
In 2009, there was a well-received talk at the O'Reilly Velocity Conference: "10+ Deploys per Day: Dev and Ops Cooperation at Flickr." Patrick then decided to organize an event in Ghent, Belgium, called DevOpsDays. This time, there was much interest, and the conference was a success. The name "DevOpsDays" struck a chord, and the conference has become a recurring event. DevOpsDays was abbreviated to "DevOps" in conversations on Twitter and various Internet forums.
The DevOps movement has its roots in Agile software development principles. The Agile Manifesto was written in 2001 by a number of individuals wanting to improve the then current status quo of system development and find new ways of working in the software development industry. The following is an excerpt from the Agile Manifesto, the now classic text, which is available on the Web at http://agilemanifesto.org/:
"Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more."
In light of this, DevOps can be said to relate to the first principle, "Individuals and interactions over processes and tools."
This might be seen as a fairly obviously beneficial way to work—why do we even have to state this obvious fact? Well, if you have ever worked in any large organization, you will know that the opposite principle seems to be in operation instead. Walls between different parts of an organization tend to form easily, even in smaller organizations, where at first it would appear to be impossible for such walls to form.
DevOps, then, tends to emphasize that interactions between individuals are very important, and that technology might possibly assist in making these interactions happen and tear down the walls inside organizations. This might seem counterintuitive, given that the first principle favors interaction between people over tools, but my opinion is that any tool can have several effects when used. If we use the tools properly, they can facilitate all of the desired properties of an Agile workplace.
A very simple example might be the choice of systems used to report bugs. Quite often, development teams and quality assurance teams use different systems to handle tasks and bugs. This creates unnecessary friction between the teams and further separates them when they should really focus on working together instead. The operations team might, in turn, use a third system to handle requests for deployment to the organization's servers.
An engineer with a DevOps mindset, on the other hand, will immediately recognize all three systems as being workflow systems with similar properties. It should be possible for everyone in the three different teams to use the same system, perhaps tweaked to generate different views for the different roles. A further benefit would be smaller maintenance costs, since three systems are replaced by one.
Another core goal of DevOps is automation and Continuous Delivery. Simply put, automating repetitive and tedious tasks leaves more time for human interaction, where true value can be created.
How fast is fast?
The turnaround for DevOps processes must be fast. We need to consider time to market in the larger perspective, and simply stay focused on our tasks in the smaller perspective. This line of thought is also held by the Continuous Delivery movement.
As with many things Agile, many of the ideas in DevOps and Continuous Delivery are in fact different names of the same basic concepts. There really isn't any contention between the two concepts; they are two sides of the same coin.
DevOps engineers work on making enterprise processes faster, more efficient, and more reliable. Repetitive manual labor, which is error prone, is removed whenever possible.
It's easy, however, to lose track of the goal when working with DevOps implementations. Doing nothing faster is of no use to anyone. Instead, we must keep track of delivering increased business value.
For instance, increased communication between roles in the organization has clear value. Your product owners might be wondering how the development process is going and are eager to have a look. In this situation, it is useful to be able to deliver incremental improvements of code to the test environments quickly and efficiently. In the test environments, the involved stake holders, such as product owners and, of course, the quality assurance teams, can follow the progress of the development process.
Another way to look at it is this: If you ever feel yourself losing focus because of needless waiting, something is wrong with your processes or your tooling. If you find yourself watching videos of robots shooting balloons during compile time, your compile times are too long!
The same is true for teams idling while waiting for deploys and so on. This idling is, of course, even more expensive than that of a single individual.
While robot shooting practice videos are fun, software development is inspiring too! We should help focus creative potential by eliminating unnecessary overhead.
A death ray laser robot versus your team's productivity
The Agile wheel of wheels
There are several different cycles in Agile development, from the Portfolio level through to the Scrum and Kanban cycles and down to the Continuous Integration cycle. The emphasis on which cadence work happens in is a bit different depending on which Agile framework you are working with. Kanban emphasizes the 24-hour cycle and is popular in operations teams. Scrum cycles can be between two to four weeks and are often used by development teams using the Scrum Agile process. Longer cycles are also common and are called Program Increments, which span several Scrum Sprint cycles, in Scaled Agile Framework.
The Agile wheel of wheels
DevOps must be able to support all these cycles. This is quite natural given the central theme of DevOps: cooperation between disciplines in an Agile organization.
The most obvious and measurably concrete benefits of DevOps occur in the shorter cycles, which in turn make the longer cycles more efficient. Take care of the pennies, and the pounds will take care of themselves, as the old adage goes.
Here are some examples of when DevOps can benefit Agile cycles:
In organizations where deployments are done mostly by hand, the time to deploy can be several days. Organizations that have these inefficient deployment processes will benefit greatly from a DevOps mindset.
The Kanban cycle is 24 hours, and it's therefore obvious that the deployment cycle needs to be much faster than that if we are to succeed with Kanban.A well-designed DevOps Continuous Delivery pipeline can deploy code from being committed to the code repository to production in the order of minutes, depending on the size of the change.
Beware the cargo cult Agile fallacy
Richard Feynman was awarded the Nobel Prize for his work in the field of quantum physics in 1965. He noticed a common behavior among scientists, in which they went though all the motions of science but missed some central, vital ingredient of the scientific process. He called this behavior "cargo cult science," since it was reminiscent of the cargo cults in the Melanesian South Sea islands. These cargo cults where formed during the Second World War when the islanders watched great planes land with useful cargo. After the war stopped, the cargo also stopped coming. The islanders started simulating landing strips, doing everything just as they had observed the American military do, in order for the planes to land.
A cargo cult Agile aeroplane
We are not working in an Agile or DevOps-oriented manner simply because we have a morning stand-up where we drink coffee and chat about the weather. We don't have a DevOps pipeline just because we have a Puppet implementation that only the operations team knows anything about.
It is very important that we keep track of our goals and continuously question whether we are doing the right thing and are still on the right track. This is central to all Agile thinking. It is, however, something that is manifestly very hard to do in practice. It is easy to wind up as followers of the cargo cults.
When constructing deployment pipelines, for example, keep in mind why we are building them in the first place. The goal is to allow people to interact with new systems faster and with less work. This, in turn, helps people with different roles interact with each other more efficiently and with less turnaround.
If, on the other hand, we build a pipeline that only helps one group of people achieve their goals, for instance, the operations personnel, we have failed to achieve our basic goal.
While this is not an exact science, it pays to bear in mind that Agile cycles, such as the sprint cycle in the Scrum Agile method, normally have a method to deal with this situation. In Scrum, this is called the sprint retrospective, where the team gets together and discusses what went well and what could have gone better during the sprint. Spend some time here to make sure you are doing the right thing in your daily work.
A common problem here is that the output from the sprint retrospective isn't really acted upon. This, in turn, may be caused by the unfortunate fact that the identified problems were really caused by some other part of the organization that you don't communicate well with. Therefore, these problems come up again and again in the retrospectives and are never remedied.
If you recognize that your team is in this situation, you will benefit from the DevOps approach since it emphasizes cooperation between roles in the organization.
To summarize, try to use the mechanisms provided in the Agile methods in your methods themselves. If you are using Scrum, use the sprint retrospective mechanism to capture potential improvements. This being said, don't take the methods as gospel. Find out what works for you.
DevOps and ITIL
This section explains how DevOps and other ways of working coexist and fit together in a larger whole.
DevOps fits well together with many frameworks for Agile or Lean enterprises. Scaled Agile Framework, or SAFe® , specifically mentions DevOps. There is nearly never any disagreement between proponents of different Agile practices and DevOps since DevOps originated in the Agile environments. The story is a bit different with ITIL, though.
ITIL, which was formerly known as Information Technology Infrastructure Library, is a practice used by many large and mature organizations.
ITIL is a large framework that formalizes many aspects of the software life cycle. While DevOps and Continuous Delivery hold the view that the changesets we deliver to production should be small and happen often, at first glance, ITIL would appear to hold the opposite view. It should be noted that this isn't really true. Legacy systems are quite often monolithic, and in these cases, you need a process such as ITIL to manage the complex changes often associated with large monolithic systems.
If you are working in a large organization, the likelihood that you are working with such large monolithic legacy systems is very high.
In any case, many of the practices described in ITIL translate directly into corresponding DevOps practices. ITIL prescribes a configuration management system and a configuration management database. These types of systems are also integral to DevOps, and several of them will be described in this book.
Summary
This chapter presented a brief overview of the background of the DevOps movement. We discussed the history of DevOps and its roots in development and operations, as well as in the Agile movement. We also took a look at how ITIL and DevOps might coexist in larger organizations. The cargo cult anti-pattern was explored, and we discussed how to avoid it. You should now be able to answer where DevOps fits into a larger Agile context and the different cycles of Agile development.
We will gradually move toward more technical and hands-on subjects. The next chapter will present an overview of what the technical systems we tend to focus on in DevOps look like.
Chapter 2. A View from Orbit
The DevOps process and Continuous Delivery pipelines can be very complex. You need to have a grasp of the intended final results before starting the implementation.
This chapter will help you understand how the various systems of a Continuous Delivery pipeline fit together, forming a larger whole.
In this chapter, we will read about:
The DevOps process and Continuous Delivery – an overview
There is a lot of detail in the following overview image of the Continuous Delivery pipeline, and you most likely won't be able to read all the text. Don't worry about this just now; we are going to delve deeper into the details as we go along.
For the time being, it is enough to understand that when we work with DevOps, we work with large and complex processes in a large and complex context.
An example of a Continuous Delivery pipeline in a large organization is introduced in the following image:
While the basic outline of this image holds true surprisingly often, regardless of the organization. There are, of course, differences, depending on the size of the organization and the complexity of the products that are being developed.
The early parts of the chain, that is, the developer environments and the Continuous Integration environment, are normally very similar.
The number and types of testing environments vary greatly. The production environments also vary greatly.
In the following sections, we will discuss the different parts of the Continuous Delivery pipeline.
The developers
The developers (on the far left in the figure) work on their workstations. They develop code and need many tools to be efficient.
The following detail from the previous larger Continuous Delivery pipeline overview illustrates the development team.
Ideally, they would each have production-like environments available to work with locally on their workstations or laptops. Depending on the type of software that is being developed, this might actually be possible, but it's more common to simulate, or rather, mock, the parts of the production environments that are hard to replicate. This might, for example, be the case for dependencies such as external payment systems or phone hardware.
When you work with DevOps, you might, depending on which of its two constituents you emphasized on in your original background, pay more or less attention to this part of the Continuous Delivery pipeline. If you have a strong developer background, you appreciate the convenience of a prepackaged developer environment, for example, and work a lot with those. This is a sound practice, since otherwise developers might spend a lot of time creating their development environments. Such a prepackaged environment might, for instance, include a specific version of the Java Development Kit and an integrated development environment, such as Eclipse. If you work with Python, you might package a specific Python version, and so on.
Keep in mind that we essentially need two or more separately maintained environments. The preceding developer environment consists of all the development tools we need. These will not be installed on the test or production systems. Further, the developers also need some way to deploy their code in a production-like way. This can be a virtual machine provisioned with Vagrant running on the developer's machine, a cloud instance running on AWS, or a Docker container: there are many ways to solve this problem.
Tip
My personal preference is to use a development environment that is similar to the production environment. If the production servers run Red Hat Linux, for instance, the development machine might run CentOS Linux or Fedora Linux. This is convenient because you can use much of the same software that you run in production locally and with less hassle. The compromise of using CentOS or Fedora can be motivated by the license costs of Red Hat and also by the fact that enterprise distributions usually lag behind a bit with software versions.
If you are running Windows servers in production, it might also be more convenient to use a Windows development machine.
The revision control system
The revision control system is often the heart of the development environment. The code that forms the organization's software products is stored here. It is also common to store the configurations that form the infrastructure here. If you are working with hardware development, the designs might also be stored in the revision control system.
The following image shows the systems dealing with code, Continuous Integration, and artifact storage in the Continuous Delivery pipeline in greater detail:
For such a vital part of the organization's infrastructure, there is surprisingly little variation in the choice of product. These days, many use Git or are switching to it, especially those using proprietary systems reaching end-of-life.
Regardless of the revision control system you use in your organization, the choice of product is only one aspect of the larger picture.
You need to decide on directory structure conventions and which branching strategy to use.
If you have a great deal of independent components, you might decide to use a separate repository for each of them.
Since the revision control system is the heart of the development chain, many of its details will be covered in Chapter 5, Building the Code.
The build server
The build server is conceptually simple. It might be seen as a glorified egg timer that builds your source code at regular intervals or on different triggers.
The most common usage pattern is to have the build server listen to changes in the revision control system. When a change is noticed, the build server updates its local copy of the source from the revision control system. Then, it builds the source and performs optional tests to verify the quality of the changes. This process is called Continuous Integration. It will be explored in more detail in Chapter 5, Building the Code.
Unlike the situation for code repositories, there hasn't yet emerged a clear winner in the build server field.
In this book, we will discuss Jenkins, which is a widely used open source solution for build servers. Jenkins works right out of the box, giving you a simple and robust experience. It is also fairly easy to install.
The artifact repository
When the build server has verified the quality of the code and compiled it into deliverables, it is useful to store the compiled binary artifacts in a repository. This is normally not the same as the revision control system.
In essence, these binary code repositories are filesystems that are accessible over the HTTP protocol. Normally, they provide features for searching and indexing as well as storing metadata, such as various type identifiers and version information about the artifacts.
In the Java world, a pretty common choice is Sonatype Nexus. Nexus is not limited to Java artifacts, such as Jars or Ears, but can also store artifacts of the operating system type, such as RPMs, artifacts suitable for JavaScript development, and so on.
Amazon S3 is a key-value datastore that can be used to store binary artifacts. Some build systems, such as Atlassian Bamboo, can use Amazon S3 to store artifacts. The S3 protocol is open, and there are open source implementations that can be deployed inside your own network. One such possibility is the Ceph distributed filesystem, which provides an S3-compatible object store.
Package managers, which we will look at next, are also artifact repositories at their core.
Package managers
Linux servers usually employ systems for deployment that are similar in principle but have some differences in practice.
Red Hat-like systems use a package format called RPM. Debian-like systems use the .deb format, which is a different package format with similar abilities. The deliverables can then be installed on servers with a command that fetches them from a binary repository. These commands are called package managers.
On Red Hat systems, the command is called yum, or, more recently, dnf. On Debian-like systems, it is called aptitude/dpkg.
The great benefit of these package management systems is that it is easy to install and upgrade a package; dependencies are installed automatically.
If you don't have a more advanced system in place, it would be feasible to log in to each server remotely and then type yum upgrade on each one. The newest packages would then be fetched from the binary repository and installed. Of course, as we will see, we do indeed have more advanced systems of deployment available; therefore, we won't need to perform manual upgrades.
Test environments
After the build server has stored the artifacts in the binary repository, they can be installed from there into test environments.
The following figure shows the test systems in greater detail:
Test environments should normally attempt to be as production-like as is feasible. Therefore, it is desirable that the they be installed and configured with the same methods as production servers.
Staging/production
Staging environments are the last line of test environments. They are interchangeable with production environments. You install your new releases on the staging servers, check that everything works, and then swap out your old production servers and replace them with the staging servers, which will then become the new production servers. This is sometimes called the blue-green deployment strategy.
The exact details of how to perform this style of deployment depend on the product being deployed. Sometimes, it is not possible to have several production systems running in parallel, usually because production systems are very expensive.
At the other end of the spectrum, we might have many hundreds of production systems in a pool. We can then gradually roll out new releases in the pool. Logged-in users stay with the version running on the server they are logged in to. New users log in to servers running new versions of the software.
The following detail from the larger Continuous Delivery image shows the final systems and roles involved:
Not all organizations have the resources to maintain production-quality staging servers, but when it's possible, it is a nice and safe way to handle upgrades.
Release management
We have so far assumed that the release process is mostly automatic. This is the dream scenario for people working with DevOps.
This dream scenario is a challenge to achieve in the real world. One reason for this is that it is usually hard to reach the level of test automation needed in order to have complete confidence in automated deploys. Another reason is simply that the cadence of business development doesn't always the match cadence of technical development. Therefore, it is necessary to enable human intervention in the release process.
A faucet is used in the following figure to symbolize human interaction—in this case, by a dedicated release manager.
How this is done in practice varies, but deployment systems usually have a way to support how to describe which software versions to use in different environments.
The integration test environments can then be set to use the latest versions that have been deployed to the binary artifact repository. The staging and production servers have particular versions that have been tested by the quality assurance team.
Scrum, Kanban, and the delivery pipeline
How does the Continuous Delivery pipeline that we have described in this chapter support Agile processes such as Scrum and Kanban?
Scrum focuses on sprint cycles, which can occur biweekly or monthly. Kanban can be said to focus more on shorter cycles, which can occur daily.
The philosophical differences between Scrum and Kanban are a bit deeper, although not mutually exclusive. Many organizations use both Kanban and Scrum together.
From a software-deployment viewpoint, both Scrum and Kanban are similar. Both require frequent hassle-free deployments. From a DevOps perspective, a change starts propagating through the Continuous Delivery pipeline toward test systems and beyond when it is deemed ready enough to start that journey. This might be judged on subjective measurements or objective ones, such as "all unit tests are green."
Our pipeline can manage both the following types of scenarios:
So, again, from a DevOps perspective, it doesn't really matter if we use Scrum, Scaled Agile Framework, Kanban, or another method within the lean or Agile frameworks. Even a traditional Waterfall process can be successfully managed—DevOps serves all!
Wrapping up – a complete example
So far, we have covered a lot of information at a cursory level.
To make it more clear, let's have a look at what happens to a concrete change as it propagates through the systems, using an example:
The process is then repeated as needed. As you can see, there is a lot going on!
Identifying bottlenecks
As is apparent from the previous example, there is a lot going on for any change that propagates through the pipeline from development to production. It is important for this process to be efficient.
As with all Agile work, keep track of what you are doing, and try to identify problem areas.
When everything is working as it should, a commit to the code repository should result in the change being deployed to integration test servers within a 15-minute time span.
When things are not working well, a deploy can take days of unexpected hassles. Here are some possible causes:
We will examine these challenges further in the chapters ahead.
Summary
In this chapter, we delved further into the different types of systems and processes you normally work with when doing DevOps work. We gained a deeper, detailed understanding of the Continuous Delivery process, which is at the core of DevOps.
Next, we will look into how the DevOps mindset affects software architecture in order to help us achieve faster and more precise deliveries.
Chapter 3. How DevOps Affects Architecture
Software architecture is a vast subject, and in this book, we will focus on the aspects of architecture that have the largest effects on Continuous Delivery and DevOps and vice versa.
In this chapter, we will see:
We finally conclude with some practical issues regarding database migration.
It's quite a handful, so let's get started!
Introducing software architecture
We will discuss how DevOps affects the architecture of our applications rather than the architecture of software deployment systems, which we discuss elsewhere in the book.
Often while discussing software architecture, we think of the non-functional requirements of our software. By non-functional requirements, we mean different characteristics of the software rather than the requirements on particular behaviors.
A functional requirement could be that our system should be able to deal with credit card transactions. A non-functional requirement could be that the system should be able to manage several such credit cards transactions per second.
Here are two of the non-functional requirements that DevOps and Continuous Delivery place on software architecture:
The normal case should be that we are able to deploy small changes all the way from developers' machines to production in a small amount of time. Rolling back a change because of unexpected problems caused by it should be a rare occurrence.
So, if we take out the deployment systems from the equation for a while, how will the architecture of the software systems we deploy be affected?
The monolithic scenario
One way to understand the issues that a problematic architecture can cause for Continuous Delivery is to consider a counterexample for a while.
Let's suppose we have a large web application with many different functions. We also have a static website inside the application. The entire web application is deployed as a single Java EE application archive. So, when we need to fix a spelling mistake in the static website, we need to rebuild the entire web application archive and deploy it again.
While this might be seen as a silly example, and the enlightened reader would never do such a thing, I have seen this anti-pattern live in the real world. As DevOps engineers, this could be an actual situation that we might be asked to solve.
Let's break down the consequences of this tangling of concerns. What happens when we want to correct a spelling mistake? Let's take a look:
Okay, this doesn't seem altogether too bad at first glance. But consider the following too:
The point here is that we have already spent considerable mental energy in making sure that the change is really safe. The system is so complex that it becomes difficult to think about the effects of changes, even though they might be trivial.
Now, a change is usually more complex than a simple spelling correction. Thus, we need to exercise all aspects of the deployment chain, including manual verification, for all changes to a monolith.
We are now in a place that we would rather not be.
Architecture rules of thumb
There are a number of architecture rules that might help us understand how to deal with the previous undesirable situation.
The separation of concerns
The renowned Dutch computer scientist Edsger Dijkstra first mentioned his idea of how to organize thought efficiently in his paper from 1974, On the role of scientific thought.
He called this idea "the separation of concerns". To this date, it is arguably the single most important rule in software design. There are many other well-known rules, but many of them follow from the idea of the separation of concerns. The fundamental principle is simply that we should consider different aspects of a system separately.
The principle of cohesion
In computer science, cohesion refers to the degree to which the elements of a software module belong together.
Cohesion can be used as a measure of how strongly related the functions in a module are.
It is desirable to have strong cohesion in a module.
We can see that strong cohesion is another aspect of the principle of the separation of concerns.
Coupling
Coupling refers to the degree of dependency between two modules. We always want low coupling between modules.
Again, we can see coupling as another aspect of the principle of the separation of concerns.
Systems with high cohesion and low coupling would automatically have separation of concerns, and vice versa.
Back to the monolithic scenario
In the previous scenario with the spelling correction, it is clear that we failed with respect to the separation of concerns. We didn't have any modularization at all, at least from a deployment point of view. The system appears to have the undesirable features of low cohesion and high coupling.
If we had a set of separate deployment modules instead, our spelling correction would most likely have affected only a single module. It would have been more apparent that deploying the change was safe.
How this should be accomplished in practice varies, of course. In this particular example, the spelling corrections probably belong to a frontend web component. At the very least, this frontend component can be deployed separately from the backend components and have their own life cycle.
